root/campo-sirio
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
campo-sirio/libraries/mcpp/doc/cpp-test.html
Alessandro Bonazzi e075990ed3 Patch level : 12.0 no-patch 
Files correlati     :
Commento            :

Aggiunto il preprocessore c++ mcpp per sostituire il compilatore nella compilazione delle maschere.
2020-11-28 16:24:08 +01:00

4089 lines
357 KiB
HTML
Raw Blame History

<html>
<head>
<meta http-equiv="Content-Type" content="text/html">
<style> tt {color: maroon} </style>
<style> table {border-collapse: separate; border-spacing: 0px; empty-cells: show; background-color: #f0f0ff} </style>
<style> th, td {text-align: left; padding-left: 15px; padding-right: 15px} </style>
</head>
<body>
<div align="center">
<h1>CPP-TEST</h1>
<h3>== Validation Suite for Standard C Conformance of Preprocessing ==</h3>
</div>
<div align="right">
<h4>for V.1.5.5 (2008/11)<br>
Kiyoshi Matsui (kmatsui@t3.rim.or.jp)</h4>
</div>
<div align="center">
<h2>Contents</h2>
</div>
<h4><dl><dt><a name="toc.1" href="#1">1. Standard C and Validation Suite</a>
<dd><a name="toc.1.1" href="#1.1">1.1. History</a>
<dd><a name="toc.1.2" href="#1.2">1.2. The Standards</a>
<dd><a name="toc.1.3" href="#1.3">1.3. Notations in this document</a>
<br>
<br>
<dt><a name="toc.2" href="#2">2. Standard C Preprocessing Features</a>
<dd><a name="toc.2.1" href="#2.1">2.1. K&R 1st and Standard C Preprocessing</a>
<dd><dl><dt><a name="toc.2.2" href="#2.2">2.2. Translation Phases</a>
<dd><a name="toc.2.2.1" href="#2.2.1">2.2.1. Line Splicing Before Tokenization</a></dl>
<dd><dl><dt><a name="toc.2.3" href="#2.3">2.3. Preprocessing Token</a>
<dd><a name="toc.2.3.1" href="#2.3.1">2.3.1. No Keyword</a>
<dd><a name="toc.2.3.2" href="#2.3.2">2.3.2. Preprocessing-number</a>
<dd><a name="toc.2.3.3" href="#2.3.3">2.3.3. Token-based Operations and Token Concatenation</a></dl>
<dd><a name="toc.2.4" href="#2.4">2.4. Evaluation Type of #if Expression</a>
<dd><a name="toc.2.5" href="#2.5">2.5. Portable Preprocessor</a>
<dd><dl><dt><a name="toc.2.6" href="#2.6">2.6. Function-like Macro Expansion</a>
<dd><a name="toc.2.6.1" href="#2.6.1">2.6.1. Analogous to Function Call</a>
<dd><dl><dt><a name="toc.2.6.2" href="#2.6.2">2.6.2. Argument Expansion Before Substitution</a>
<dd><a name="toc.2.6.2.1" href="#2.6.2.1">2.6.2.1. No Expansion for the Operand of the # or ## Operator</a></dl>
<dd><a name="toc.2.6.3" href="#2.6.3">2.6.3. Rescanning</a>
<dd><a name="toc.2.6.4" href="#2.6.4">2.6.4. Prevention of Recursive Macro Expansion</a></dl>
<dd><dl><dt><a name="toc.2.7" href="#2.7">2.7. Issues</a>
<dd><a name="toc.2.7.1" href="#2.7.1">2.7.1. Header Name in the &lt;stdio.h&gt; Format</a>
<dd><a name="toc.2.7.2" href="#2.7.2">2.7.2. # Operator Specification with Legacy from Character-based Preprocessing</a>
<dd><a name="toc.2.7.3" href="#2.7.3">2.7.3. White Space Handling at Macro Re-definition</a>
<dd><a name="toc.2.7.4" href="#2.7.4">2.7.4. Parameter Name at Function-like Macro Re-definition</a>
<dd><a name="toc.2.7.5" href="#2.7.5">2.7.5. Unpredictable Evaluation of Character Constant in #if Expression</a>
<dd><a name="toc.2.7.6" href="#2.7.6">2.7.6. Non-Function-like Rescanning of Function-like Macro</a>
<dd><a name="toc.2.7.7" href="#2.7.7">2.7.7. C90 Corrigendum 1, 2 and Amendment 1</a>
<dd><a name="toc.2.7.8" href="#2.7.8">2.7.8. Redundant Specifications</a></dl>
<dd><a name="toc.2.8" href="#2.8">2.8. Preprocessing Specification in C99</a>
<dd><a name="toc.2.9" href="#2.9">2.9. Toward Clear Preprocessing Specifications</a>
<br>
<br>
<dt><a name="toc.3" href="#3">3. Validation Suite Explanation</a>
<dd><a name="toc.3.1" href="#3.1">3.1. Validation Suite for Conformance of Preprocessing</a>
<dd><dl><dt><a name="toc.3.2" href="#3.2">3.2. Testing Method</a>
<dd><a name="toc.3.2.1" href="#3.2.1">3.2.1. Manual Testing</a>
<dd><a name="toc.3.2.2" href="#3.2.2">3.2.2. Automatic Testing by cpp_test</a>
<dd><dl><dt><a name="toc.3.2.3" href="#3.2.3">3.2.3. Automatic Testing by GCC / testsuite</a>
<dd><a name="toc.3.2.3.1" href="#3.2.3.1">3.2.3.1. TestSuite</a>
<dd><a name="toc.3.2.3.2" href="#3.2.3.2">3.2.3.2. Installation to TestSuite and Testing</a>
<dd><a name="toc.3.2.3.3" href="#3.2.3.3">3.2.3.3. <b>mcpp</b> Automatic Testing</a>
<dd><a name="toc.3.2.3.4" href="#3.2.3.4">3.2.3.4. TestSuite and Validation Suite</a></dl></dl>
<dd><a name="toc.3.3" href="#3.3">3.3. Violation of syntax rules or Constraints and Diagnostic Messages</a>
<dd><dl><dt><a name="toc.3.4" href="#3.4">3.4. Details</a>
<dd><a name="toc.3.4.1" href="#3.4.1">3.4.1. Trigraphs</a>
<dd><a name="toc.3.4.2" href="#3.4.2">3.4.2. Line Splicing by &lt;backslash&gt;&lt;newline&gt;</a>
<dd><a name="toc.3.4.3" href="#3.4.3">3.4.3. Comments</a>
<dd><a name="toc.3.4.4" href="#3.4.4">3.4.4. Special Tokens (digraphs) and Characters (UCN)</a>
<dd><a name="toc.3.4.5" href="#3.4.5">3.4.5. Spaces and Tabs on a Preprocessing Directive Line</a>
<dd><a name="toc.3.4.6" href="#3.4.6">3.4.6. #include</a>
<dd><a name="toc.3.4.7" href="#3.4.7">3.4.7. #line</a>
<dd><a name="toc.3.4.8" href="#3.4.8">3.4.8. #error</a>
<dd><a name="toc.3.4.9" href="#3.4.9">3.4.9. #pragma, _Pragma() operator</a>
<dd><a name="toc.3.4.10" href="#3.4.10">3.4.10. #if, #elif, #else, and #endif</a>
<dd><a name="toc.3.4.11" href="#3.4.11">3.4.11. #if defined</a>
<dd><a name="toc.3.4.12" href="#3.4.12">3.4.12. #if Expression Type</a>
<dd><a name="toc.3.4.13" href="#3.4.13">3.4.13. #if Expression Evaluation</a>
<dd><a name="toc.3.4.14" href="#3.4.14">3.4.14. #if Expression Error</a>
<dd><a name="toc.3.4.15" href="#3.4.15">3.4.15. #ifdef and #ifndef</a>
<dd><a name="toc.3.4.16" href="#3.4.16">3.4.16. #else and #endif Errors</a>
<dd><a name="toc.3.4.17" href="#3.4.17">3.4.17. #if, #elif, #else, and #endif Mismatching Errors</a>
<dd><a name="toc.3.4.18" href="#3.4.18">3.4.18. #define</a>
<dd><a name="toc.3.4.19" href="#3.4.19">3.4.19. Macro Re-definition</a>
<dd><a name="toc.3.4.20" href="#3.4.20">3.4.20. Macro Names Same as Keywords</a>
<dd><a name="toc.3.4.21" href="#3.4.21">3.4.21. Macro Expansion Requiring Pp-token Separation</a>
<dd><a name="toc.3.4.22" href="#3.4.22">3.4.22. Macro-like Sequence in a Pp-number</a>
<dd><a name="toc.3.4.23" href="#3.4.23">3.4.23. Macros Using the ## Operator</a>
<dd><a name="toc.3.4.24" href="#3.4.24">3.4.24. Macros Using the # Operator</a>
<dd><a name="toc.3.4.25" href="#3.4.25">3.4.25. Macro Expansion in a Macro Argument</a>
<dd><a name="toc.3.4.26" href="#3.4.26">3.4.26. Macros of a Same Name during Macro Rescanning</a>
<dd><a name="toc.3.4.27" href="#3.4.27">3.4.27. Macro Rescanning</a>
<dd><a name="toc.3.4.28" href="#3.4.28">3.4.28. Predefined Macros</a>
<dd><a name="toc.3.4.29" href="#3.4.29">3.4.29. #undef</a>
<dd><a name="toc.3.4.30" href="#3.4.30">3.4.30. Macro Calls</a>
<dd><a name="toc.3.4.31" href="#3.4.31">3.4.31. Macro Call Error</a>
<dd><a name="toc.3.4.32" href="#3.4.32">3.4.32. Character Constant in #if Expression</a>
<dd><a name="toc.3.4.33" href="#3.4.33">3.4.33. Wide Character Constant in #if Expression</a>
<dd><a name="toc.3.4.35" href="#3.4.35">3.4.35. Multi-Character Character Constant in #if Expression</a>
<dd><a name="toc.3.4.37" href="#3.4.37">3.4.37. Translation limits</a></dl>
<dd><a name="toc.3.5" href="#3.5">3.5. Documentation of Implementation-Defined Items</a>
<br>
<br>
<dt><a name="toc.4" href="#4">4. Evaluation of Aspects Unspecified by Standard</a>
<dd><a name="toc.4.1" href="#4.1">4.1. Multi-byte Character Encoding</a>
<dd><a name="toc.4.2" href="#4.2">4.2. Undefined Behavior</a>
<dd><a name="toc.4.3" href="#4.3">4.3. Unspecified Behavior</a>
<dd><a name="toc.4.4" href="#4.4">4.4. Other Cases Where a Warning is Preferable</a>
<dd><dl><dt><a name="toc.4.5" href="#4.5">4.5. Other Quality Matters</a>
<dd><a name="toc.4.5.1" href="#4.5.1">4.5.1. Qualities regarding Behaviors</a>
<dd><a name="toc.4.5.2" href="#4.5.2">4.5.2. Options and Extended Functionalities</a>
<dd><a name="toc.4.5.3" href="#4.5.3">4.5.3. Efficiency and others</a>
<dd><a name="toc.4.5.4" href="#4.5.4">4.5.4. Quality of Documents</a></dl>
<dd><a name="toc.4.6" href="#4.6">4.6. C++ Preprocessing</a>
<br>
<br>
<dt><a name="toc.5" href="#5">5. Issues Around C Preprocessing</a>
<dd><dl><dt><a name="toc.5.1" href="#5.1">5.1. Standard Header Files</a>
<dd><a name="toc.5.1.1" href="#5.1.1">5.1.1. General Rules</a>
<dd><a name="toc.5.1.2" href="#5.1.2">5.1.2. &lt;assert.h&gt;</a>
<dd><a name="toc.5.1.3" href="#5.1.3">5.1.3. &lt;limits.h&gt;</a>
<dd><a name="toc.5.1.4" href="#5.1.4">5.1.4. &lt;iso646.h&gt;</a></dl>
<br>
<dt><a name="toc.6" href="#6">6. Preprocessor Test Results</a>
<dd><a name="toc.6.1" href="#6.1">6.1. Preprocessors Tested</a>
<dd><a name="toc.6.2" href="#6.2">6.2. Lists of Marks</a>
<dd><a name="toc.6.3" href="#6.3">6.3. Characteristics of Each Preprocessor</a>
<dd><a name="toc.6.4" href="#6.4">6.4. Overall Review</a>
<dd><a name="toc.6.5" href="#6.5">6.5. Test Reports and Comments</a>
</dl>
<br>
<h1><a name="1" href="#toc.1">1. Standard C and Validation Suite</a></h1>
<p>I completely rewrote DECUS cpp by Martin Minow to create a portable C preprocessor called <b>mcpp</b> V.2. <b>mcpp</b> stands for 'Matsui cpp'. This preprocessor is provided as source code and can be ported for various compiler systems by modifying some macros in header files on compilation. In addition, execution programs has various behavioral modes such as Standard C (ISO/ANSI/JIS C) and others. Among those modes, the Standard C mode literally implements strict Standard C preprocessing.</p>
<p>While implementing this preprocessor, I also created a testing tool called "Validation Suite for Standard C Conformance of Preprocessing". This suite also covers C++. This document explains the Validation Suite. The Validation Suite is open to the public as an open-source-software along with this documentation under the BSD-style-license.</p>
<br>
<h2><a name="1.1" href="#toc.1.1">1.1. History</a></h2>
<p>The Validation Suite became available to the public on NIFTY SERVE/FC/LIB 2 in August 1998, and also later on http://www.vector.co.jp/pack. It did not have a version number, however, and so is assumed to be version 1.0.</p>
<p>V.1.1 supports the C99 draft in August 1997 and is an update to V.1.0. V.1.1 was also made public on NIFTY SERVE/FC/LIB 2 and vector/software pack in September, 1998.</p>
<p>V.1.2 supports the official C++ Standard release and is a small update to V.1.1. It also became available on NIFTY SERVE/FC/LIB 2 and vector/software pack in November 1998.</p>
<p>V.1.3 supports the official C99 release and is an update to V.1.2. In addition, behavioral test samples were rewritten so that they can be used by the GCC / testsuite.</p>
<p>V.1.3, while it was under development, was adopted as the year 2002's "Exploratory Software Project" at the Information-technology Promotion Agency, Japan (IPA) by Yutaka Niibe Project Manager. From July 2002 to February 2003, the development continued under the grants-in-aid from IPA and Niibe PM's advice. The English version of the document was created under my supervision with the translation work outsourced to HighWell, Inc. In February 2003, <b>mcpp</b> V.2.3 and Validation Suite V.1.3 were released on m17n.org.</p>
<p>In addition, <b>mcpp</b> and Validation Suite were adopted as the year 2003's "Exploratory Software Project" by Hiroshi Ichiji Project Manager. This allowed an update to V.2.4 and V.1.4. *1</p>
<p><b>mcpp</b> and Validation Suite have been kept on updating after the project. V.2.5 and V.1.5 were released in March 2005. Validation Suite V.1.5 changed allocation of points and some other matters. In July 2006, <b>mcpp</b> V.2.6 and Validation Suite V.1.5.1 are released. Validation Suite V.1.5.1 updated the test result of the preprocessors. In November 2006, <b>mcpp</b> V.2.6.2 and Validation Suite V.1.5.2 were released. In April 2007, <b>mcpp</b> V.2.6.3 and Validation Suite V.1.5.3 were released. V.1.5.2 and V.1.5.3 of Validation Suite have no substantial change. They have only small updates of this document.
In May 2007, <b>mcpp</b> V.2.6.4 was released, and in March 2008, <b>mcpp</b> V.2.7 and Validation Suite V.1.5.4 were released. Validation Suite V.1.5.4 updated a few testcases and added test data on Visual C++ 2008.
In May 2008, <b>mcpp</b> V.2.7.1 was released, and in November 2008, <b>mcpp</b> V.2.7.2 and Validation Suite V.1.5.5 were released. Validation Suite V.1.5.5 added test data on Wave V.2.0.</p>
<p>Note:</p>
<p>*1 The overview of the Exploratory Software Project can be found below (in Japanese only).</p>
<blockquote>
<p><a href="http://www.ipa.go.jp/jinzai/esp/">http://www.ipa.go.jp/jinzai/esp/</a></p>
</blockquote>
<p><b>mcpp</b> from V.2.3 through V.2.5 had been located at:</p>
<blockquote>
<p>http://www.m17n.org/mcpp/</p>
</blockquote>
<p>In April 2006, <b>mcpp</b> project moved to:</p>
<blockquote>
<p><a href="http://mcpp.sourceforge.net/">http://mcpp.sourceforge.net/</a></p>
</blockquote>
<p><b>mcpp</b> V.2.2 and Validation Suite V.1.2 are located in the following Vector's web site. They are in the directory called dos/prog/c, but they are not for MS-DOS exclusively. Sources are for UNIX, WIN32, MS-DOS. The documents are Japanese only.</p>
<blockquote>
<a href="http://www.vector.co.jp/soft/dos/prog/se081188.html">http://www.vector.co.jp/soft/dos/prog/se081188.html</a><br>
<a href="http://www.vector.co.jp/soft/dos/prog/se081189.html">http://www.vector.co.jp/soft/dos/prog/se081189.html</a><br>
<a href="http://www.vector.co.jp/soft/dos/prog/se081186.html">http://www.vector.co.jp/soft/dos/prog/se081186.html</a><br>
</blockquote>
<p>The text files in these archive files available at Vector use [CR]+[LF] as a &lt;newline&gt; and encode Kanji in shift-JIS for DOS/Windows. On the other hand, those from V.2.3 through V.2.5 available at SourceForge use [LF] as a &lt;newline&gt; and encode Kanji in EUC-JP for UNIX. From V.2.6 on two types of archive, .tar.gz file with [LF]/EUC-JP and .zip file with [CR]+[LF]/shift-JIS, are provided.</p>
<br>
<h2><a name="1.2" href="#toc.1.2">1.2. The Standards</a></h2>
<p>ISO/IEC 9899:1990 (JIS X 3010-1993) had been used as C Standard, but in 1999, ISO/IEC 9899:1999 was adopted as a new Standard. This document calls the former C90 and latter C99. The former is generally called ANSI C or C89 because it migrated from ANSI X3.159-1989. ISO/IEC 9899:1990 plus its Amendment 1995 is sometimes called C95. C++ Standards are ISO/IEC 14882:1998 and its corrigendum version ISO/IEC 14882:2003. This document calls both of them C++98.</p>
<p>The Standards referred in this explanation are below.</p>
<pre>
C90:
ANSI X3. 159-1989 (ANSI, New York, 1989)
ISO/IEC 9899:1990(E) (ISO/IEC, Switzerland, 1990)
ibid. Technical Corrigendum 1 (ibid., 1994)
ibid. Amendment 1: C Integrity (ibid., 1995)
ibid. Technical Corrigendum 2 (ibid., 1996)
JIS X 3010-1993 (JIS Handbook 59-1994, Tokyo, 1994, Japanese Standards Association)
C99:
ISO/IEC 9899:1999(E)
ibid. Technical Corrigendum 1 (2001)
ibid. Technical Corrigendum 2 (2004)
C++:
ISO/IEC 14882:1998(E)
</pre>
<p>ANSI X3.159 contained "Rationale." It was not adopted by ISO C90 for some reason, but reappeared in ISO C99. This "Rationale" is also referred to from time to time.</p>
<p>PDF versions of C99 and C++ Standards can be obtained online on the following sites. (The open-std.org site provides drafts which can be freely downloaded.)</p>
<p>C99, C++98, C++03:</p>
<blockquote>
<p><a href="http://webstore.ansi.org/ansidocstore/default.asp">http://webstore.ansi.org/ansidocstore/default.asp</a></p>
</blockquote>
<p>C99+TC1+TC2:</p>
<blockquote>
<p><a href="http://www.open-std.org/jtc1/sc22/wg14/www/docs/n1124.pdf">http://www.open-std.org/jtc1/sc22/wg14/www/docs/n1124.pdf</a></p>
</blockquote>
<p>C99 Rationale final draft in October, 1999:</p>
<blockquote>
<p><a href="http://www.open-std.org/jtc1/sc22/wg14/www/docs/n897.pdf">http://www.open-std.org/jtc1/sc22/wg14/www/docs/n897.pdf</a></p>
</blockquote>
<br>
<h2><a name="1.3" href="#toc.1.3">1.3. Notations in this document</a></h2>
<p>Though this document was text-file in the older versions, it has changed to html-file at V.1.5.2. This document uses the following typographical conventions:</p>
<ul>
<li><tt style="color:navy">source</tt>:<br>
<tt style="color:navy">Navy</tt> colored constant-width font is used for code snippets, command line inputs and program outputs.<br>
<li><tt>__STDC__</tt>:<br>
<tt>Maroon</tt> colored constant-width font is used for Standard predefined macros.<br>
</ul>
<br>
<h1><a name="2" href="#toc.2">2. Standard C Preprocessing Features</a></h1>
<p>Before explaining the Validation Suite, I will talk about the overall characteristics of Standard C (ANSI/ISO/JIS C) preprocessing. This is not text bookish, but intended to point out the concepts and issues of Standard C by comparing with K&amp;R 1st.</p>
<p>As I explain, I will concentrate on the differences between K&amp;R 1st and C90 first, C90 and C99, then C90 and C++ in this order. C99 is currently available as a Standard, however, it has not been implemented on actual compiler systems very much. Therefore, it is more realistic to center on C90.</p>
<p>This chapter shows few samples, please refer to the Validation Suite since it is a sample itself.</p>
<br>
<h2><a name="2.1" href="#toc.2.1">2.1. K&amp;R 1st and Standard C Preprocessing</a></h2>
<p>There were endless varieties of dialects amongst pre-Standard C language implementations. Above all, there were almost no standards for preprocessing. The reason was because the preprocessing specification was too simplistic and ambiguous in the 1st edition of "The C Programming Language" by Kernighan &amp; Ritchie as a reference. In addition, it seems that preprocessing was thought to be a bonus to the language proper. However, many features were added to preprocessing by each implementation since K&amp;R 1st. Some supplemented flaws in the language proper while others tried to maintain portability among different implementations. However, there were too many differences among implementations in any case. The truth was it was nowhere close to being portable.</p>
<p>C90 provided a clear specification on preprocessing which had been a cause of the confusion for many years. There are some new features added which are well known. What is more important, however, is that C90 provides virtually the first overall specification on preprocessing. You can see a basic point of view, "what is preprocessing?", which had been vague thus far, everywhere in this specification. Preprocessing in C90 is not just K&amp;R 1st + alpha. In order to understand this, I believe we need to grasp not only "new features", but also such basics clearly. Unfortunately, however, the basics of preprocessing are not summarized together in the body of the Standard and just mentioned briefly in "Rationale", a commentary on the Standard. Even more unfortunately, it contains incoherent parts which seem to be the results of making a compromise with conventional preprocessing. Therefore, I will summarize basic characteristics of C90 preprocessing and examine their issues.</p>
<p>Characteristics different from pre-Standard processing or newly defined are summarized as the following four points.</p>
<ol>
<li>Literal "pre-processing" independent of implementation-specific parts in the language proper (execution environment, so to speak.) It is extremely rare for preprocessing to end in an unexpected result depending on implementation. This allows us to write portable source code for a preprocessor itself. Also, only one preprocessor executable program suffices for each OS.<br>
<br>
<li>The translation phase specification clearly defines a procedure for tokenizing source. A token is handled in an interim form as a preprocessing token until preprocessing completes. Specifying preprocessing tokens and tokens themselves separately helps preprocessing getting independent of implementation-dependent parts in the language proper.<br>
<br>
<li>Preprocessing takes place in the unit of preprocessing token and is token-oriented in principle. On the contrary, pre-Standard preprocessing was supposedly token-oriented but contained parts for character-oriented processing to no small extent based on its historical background; it was neither one thing nor the other.<br>
<br>
<li>Function-like macro expansion is modeled after a function call to organize grammar. Function-like macro calls can be used anywhere function calls can. Processing for a macro call in an argument parallels the evaluation for a function call in a function argument and is substituted for a parameter in the replacement list after the macro in the argument is completely expanded. At this expansion, the macro call in the argument must be completed within the argument.<br>
</ol>
<p>These principles are examined below in turn.</p>
<br>
<h2><a name="2.2" href="#toc.2.2">2.2. Translation Phases</a></h2>
<p>No preprocessing procedure was described in K&amp;R 1st, which caused much confusion. C90 specifies and defines translation phases. It can be summarized as follows. *1</p>
<ol>
<li>Map the characters in source files to the source character set if necessary. Replace trigraphs. *2<br>
<br>
<li>Delete &lt;backslash&gt;&lt;newline&gt;. By doing so, splice physical lines to form logical lines.<br>
<br>
<li>Decompose source files into preprocessing tokens and white spaces. Comments are replaced by one space character. &lt;newline&gt; is retained.<br>
<br>
<li>Execute preprocessing directives and expand macro calls. If #include directive exists, files specified are processed for phase 1 to phase 4 recursively.<br>
<br>
<li>Convert characters in the source character set into the execution character set. Convert escape sequences in character constant and string literals similarly.<br>
<br>
<li>Concatenate adjacent string literals and concatenate adjacent wide character string literals.<br>
<br>
<li>Convert preprocessing tokens into tokens and compile.<br>
<br>
<li>Link.<br>
</ol>
<p>Needless to say, these steps do not actually have to be in separate phases as long as they are processed to lead the same result.</p>
<p>Among these phases, phase 1 to 4 or to 6 belong to the range of preprocessing. It is usually handled up to phase 4 since token separators such as &lt;newline&gt; need to be retained if a preprocessor is an independent program and outputs the preprocessing result as an intermediate file (if escape sequences such as \n are converted in phase 5, they cannot be distinguished from &lt;newline&gt; and other token separators.) This Validation Suite also tests up to phase 4.</p>
<p>Note:</p>
<pre>
*1 C90 5.1.1.2 Translation phases
C99 5.1.1.2 Translation phases
</pre>
C99 added _Pragma() operator processing at phase 4. Some words have been added to the description, but no changes to the meanings in particular.</p>
<p>*2 In the C99 draft in November, 1997, a multi-byte character not included in the basic source character set before trigraphs replacement was, by a specification, to be converted into a Unicode hexadecimal sequence of the \Uxxxxxxxx format, that is, a universal character name. This sequence is re-converted into the execution character set in phase 6. This is similar to the C++ Standard.</p>
<p>This specification is vague, furthermore, the load is great on implementations. Fortunately, the processing at phase 1 was deleted in the draft in January, 1999 and remained as is in C99 official version.</p>
<h3><a name="2.2.1" href="#toc.2.2.1">2.2.1. Line Splicing Before Tokenization</a></h3>
<p>K&amp;R 1st described only 2 cases below regarding line splicing by &lt;backslash&gt;&lt;newline&gt;, no others were defined.</p>
<ol>
<li>In the middle of a long #define line<br>
<li>In the middle of a long string literal<br>
</ol>
<p>C90 clearly defines that &lt;backslash&gt;&lt;newline&gt; is deleted in phase 2 before decomposing lines into preprocessing tokens and token separators in phase 3. This allows any lines or any tokens to be spliced.</p>
<p>Also, as trigraph processing is done in phase 1, the ??/&lt;newline&gt; sequence is similarly deleted. On the other hand, 0x5c code in the second byte of Kanji characters is not a &lt;backslash&gt; for the implementation with ASCII as its basic character set and Shift-JIS Kanji Characters as the multi-byte character encoding since one Kanji character is one multi-byte character.</p>
<p>It is good that the translation phases became clear, but I am wondering if it is necessary to support line splicing in the middle of a token. Although this was the only way to write a long string literal which cannot fit on one line on the screen in K&amp;R 1st, it is not necessary to start a new line in the middle of a token on purpose since adjacent string literals are concatenated in C90. Line splicing is required only when you write a long control line. If that were the only issue, it would have been better to reverse phase 2 and 3.</p>
<p>Still, the current specification seems to be adopted for backward compatibility so that the source written assuming the specification of the concatenation by line splicing for string literals in K&amp;R 1st can be processed. The specification is almost meaningless for new source in practical use, however, it is probably appropriate since it is simple, comprehensible, and easiest to implement.</p>
<br>
<h2><a name="2.3" href="#toc.2.3">2.3. Preprocessing Token</a></h2>
<p>The concept of preprocessing token (pp-token, for short) was also introduced for the first time in C90. Since it does not seem to be known very much, however, I will summarize the content at first. Below are specified as pp-tokens. *1</p>
<blockquote>header-name<br>
identifier<br>
preprocessing-number<br>
character-constant<br>
string-literal<br>
operator<br>
punctuator<br>
Non-white-space character other than above</blockquote>
<p>These look nothing special at a casual glance, but they are quite different from the token proper. Tokens are below.</p>
<blockquote>keyword<br>
identifier<br>
constant (floating-constant, integer-constant, enumeration-constant, character-constant)<br>
string-literal<br>
operator<br>
punctuator</blockquote>
<p>Pp-tokens differ from tokens in the following points.</p>
<ol>
<li>No keywords. A name the same as a keyword is handled as an identifier.<br>
<li>For constants, character-constant is same while floating-constants, integer-constants, and enumeration constants do not exist and preprocessing-numbers replace floating-constants and integer-constants.<br>
<li>Header-name exists only as a pp-token.<br>
<li>Operator and punctuator are almost the same, but # and ## operators and # punctuator exist only as a pp-token. (All are valid only in preprocessing directive lines.)<br>
</ol>
<p>Surprisingly, only string-literal and character-constants are the same. The most important of all are no keywords and the existence of a preprocessing-number instead of a numeric token. We will discuss these 2 items further.</p>
<p>Note:</p>
<pre>
*1 C90 6.1 Lexical elements
C99 6.4 Lexical elements
</pre>
<p>In C99, an operator was absorbed by a punctuator whether it is a pp-token or token. Operator became the term simply for an "operator" functionality, not as a token type. The same punctuator token (punctuator pp-token) may function as a punctuator or operator depending on the context. Also, _Pragma was added as a pp-token operator.</p>
<h3><a name="2.3.1" href="#toc.2.3.1">2.3.1. No Keyword</a></h3>
<p>A keyword is recognized for the first time in phase 7. A keyword is handled as an identifier in preprocessing phases. For preprocessing, an identifier is a macro name or an identifier which has not been defined as a macro. That means that even a macro with a same name as a keyword can be used. *1</p>
<p>This specification is indispensable in order to separate preprocessing from implementation-dependencies. This, for example, prohibits using a cast or sizeof in #if expressions. *2</p>
<p>Note:</p>
<p>*1 To be more precise, a parameter name in a macro definition is also an identifier. In addition, a preprocessing directive name is a special identifier and has a similar characteristic to a keyword. Whether this is a directive, however, is judged by syntax. If it is not in a valid place for directives, it is simply an identifier, which may be subject to macro expansion.</p>
<p>*2 Refer to <a href="#3.4.14.7">3.4.14.7</a> and <a href="#3.4.14.8">3.4.14.8</a>.</p>
<h3><a name="2.3.2" href="#toc.2.3.2">2.3.2. Preprocessing-number</a></h3>
<p>Preprocessing-numbers (pp-number, for short) are specified as below. *1, *2</p>
<blockquote>
digit<br>
.digit<br>
pp-number digit<br>
pp-number nondigit<br>
pp-number e sign<br>
pp-number E sign<br>
pp-number .<br>
</blockquote>
<p>Non-digits are letters and underscores.</p>
<p>Summarized as below.</p>
<ol>
<li>Starts with a digit or .digit.<br>
<li>The rest is sequences of letters (alphabets), underscores, digits, periods, and e+, e-, E+, or E- in any order.<br>
</ol>
<p>Pp-number includes all floating-constants and integer-constants, and even non-numerical sequences, 3E+xy, for example. Pp-number was adopted to make preprocessing simple and is considered to be helpful for the tokenization of this type of sequence which precedes semantic interpretation. *3</p>
<p>It is correct that tokenization becomes simple, however, a non-numeric pp-numbers is not a valid token. Therefore, it must disappear before completing preprocessing. Use of non-numeric pp-numbers deliberately in source is highly unlikely. The only case I can think of is that a numeric pp-number and another type of pp-token are concatenated to become a non-numeric pp-number in the macro defined by using a ## operator and it is stringized by a macro defined by a # operator. Any pp-token becomes a valid token if it is put in a string literal. However, without accepting the existence of non-numeric pp-number, the one generated by concatenation will not become a valid pp-token (the result becomes undefined.)</p>
<p>Although this type of usage is extremely special and need not to be examined in detail, pp-numbers provide an interesting subject matter in terms of token-oriented preprocessing.</p>
<p>Note:</p>
<pre>
*1 C90 6.1.8 Preprocessing numbers
C99 6.4.8 Preprocessing numbers
</pre>
<p>*2 In C99, pp-number p sign and pp-number P sign sequences were added to enable hexadecimal expression for floating point numbers. In addition, the nondigit above was replaced by the identifier-nondigit. This is a change accompanied by the approval of using an UCN (universal character name) and implementation-defined multi-byte characters in an identifier. (Refer to <a href=#2.8>2.8</a>.) In other words, an UCN can be used in a pp-number and an implementation using multi-byte characters is supported. This is allowed in case of stringizing by ## and # operators though no UCN or multi-byte characters are included in a numerical value token.</p>
<pre>
*3 C89 Rationale 3.1.8 Preprocessing numbers
C99 Rationale 6.4.8 Preprocessing numbers
</pre>
<h3><a name="2.3.3" href="#toc.2.3.3">2.3.3. Token-based Operations and Token Concatenation</a></h3>
<p>C90 acquired pp-token concatenation capability with the ## binary operator in a macro definition. This is known as a "new feature" in C90. This, however, is something introduced to replace a hack rather than a new feature. What I would like to pay attention here is that this is essential for token-oriented preprocessing.</p>
<p>The traditional token concatenation method, uses a specification of replacement of a comment with 0 character, known as the so-called "Reiser type cpp." On other occasions, token concatenation occurs unintentionally in the preprocessor with character-oriented operations. And, there were hacks taking advantage of it. I can say all took advantage of flaws in character-oriented preprocessing.</p>
<p>On the other hand, C90 allows explicit token concatenation by token-oriented operations. Source file is decomposed into sequences of pp-tokens and white spaces in translation phase 3. The only cases of combining pp-tokens later is ## operator concatenation and # operator stringizing, header-name construction, and the concatenation of string literals or wide character string literals next to each other. The handling of non-numeric pp-numbers is clear if its existence is considered in this context. That is to say, there are the following principles of C90 tokenization.</p>
<p>Pp-tokens are not concatenated implicitly. Concatenation must be done explicitly by ## operators. Pp-tokens concatenated once will never be separated again.</p>
<p>In pre-Standard character-oriented preprocessing, macro call expansion sometimes caused unintended concatenation with tokens before and after with the token sequence as its result. However, this can be thought as something which must not occur in token-oriented C90 preprocessing. *1</p>
<p>Note:</p>
<p>*1 Refer to <a href="#3.4.21">3.4.21</a>.</p>
<br>
<h2><a name="2.4" href="#toc.2.4">2.4. Evaluation Type of #if Expression</a></h2>
<p>In C90, the #if expression was evaluated with one type of size, either long or unsigned long. This also simplified preprocessing and helped reducing implementation-dependent parts at the same time. Compared with the int size, which varies greatly depending on implementations, long/unsigned long are 32 bits for the most part and sometimes 64 or 36 bit. This assures considerable portability in general #if expressions. *1, *2, *3</p>
<p>Note:</p>
<pre>
*1 C90 6.8.1 Conditional inclusion -- Semantics
*2 C99 6.10.1 Conditional inclusion
</pre>
<p>In C99, the #if expression type was specified as the maximum integer type for the compiler system. Since C99 requires long long/unsigned long long, this would be long long/unsigned long long or wider. This, however, reduces the portability of #if expressions.</p>
<p>*3 There will be more implementations with 64 bit long in the future, but I am not sure if that is good... By the way, I personally believe that the integer type size should be defined as below.</p>
<ol>
<li>short as 2 bytes and long as 4 bytes.<br>
<li>longlong (not long long) or quadra as 8 bytes.<br>
<li>int should be CPU dependent. In other words, either short, long, or longlong.<br>
<li>Stop using the usage of short int and long int by modifying int with short or long.<br>
</ol>
<p>In other words, there has been a constraint of sizeof (short) &lt;= sizeof (int) &lt;= sizeof (long) since the arrival of 64 bit compiler systems which made everything constrained and caused no type to maintain portability. It will be better to remove this constraint and to decide types by absolute size.</p>
<br>
<h2><a name="2.5" href="#toc.2.5">2.5. Portable Preprocessor</a></h2>
<p>Standard C preprocessing specifications above allow writing the source code for a preprocessor itself portably. That is because the preprocessor needs to know nothing about the implementation-dependent parts of compiler proper. Only peripheral parts as below become problems when you actually try to write preprocessors portably in Standard C compiler systems.</p>
<ol>
<li>Path list description format of OS for #include processing. Where standard header files exist.<br>
<li>Format to pass filename and line number information to a compiler proper.<br>
<li>Runtime options.<br>
<li>Character set.<br>
<li>The size of long/unsigned long in C90 or the maximum integer type in C99 must not go below that of the target implementation in cross implementations.<br>
</ol>
<p>2 and 3 above are necessary only if they are implemented for existing implementations. I expect that 2 will be standardized in the #line 123 "filename" format same as a source file. 3 is not necessary (logically, leaving convenience aside) for Standard C preprocessing. 4 can be written in the form such that special implementations are not required depending the source code (though implementations are easier for a basic character set if a table is used in the source code.) 5 will not be a problem since the integer type size for a host implementation is seldom smaller than that of target one in reality.</p>
<p>Needless to say, <b>mcpp</b> was also created with a motive that these Standard C preprocessing specifications are independent of compiler proper (though there are many #if sections in order to assure the portability since this <b>mcpp</b> is intended to be ported to many compiler systems.)</p>
<br>
<h2><a name="2.6" href="#toc.2.6">2.6. Function-like Macro Expansion</a></h2>
<p>Macro expansion method with arguments is specified modeling after function calls in C90 and called a function-like macro. If macros are contained in an argument, they are expanded before substitution for a parameter in principle.</p>
<p>This point was not clear for pre-Standard implementations. I suspect that most used the method that an argument was substituted for a parameter without the macro within it being expanded and was expanded at rescanning. So to speak, the editor-like text replacement repetition is speculated to be behind this type of expansion. In general, text replacement repetitions for editors are fine for macro expansion without arguments, but I wonder if they were extended to the macro expansion with arguments for many preprocessors.</p>
<p>However, this method causes a strange way of using macros totally different from function call-like appearance in source. When calling a nested macro with arguments, the situation where it is not clear which argument is which occurs. Given these points, implementation-specific features will increase. In short, I can say that it became too heavy a load for the editor-like text replacement repetition as macro expansion of C grew to take on advanced features.</p>
<h3><a name="2.6.1" href="#toc.2.6.1">2.6.1. Analogous to Function Call</a></h3>
<p>In consideration of this confusion, the grammar was organized in Standard C by positioning function-like macro calls as replacements for function calls. The Rationale states the following principles for which the Standard C specifications of macros are grounded. *1</p>
<blockquote>- Allow macros to be used wherever functions can be.<br>
- Define macro expansion such that it produces the same token sequence whether the macro calls appear in open text, in macro arguments, or in macro definitions.</blockquote>
<p>This stands to reason for function calls, but not for macros with arguments. It is obvious that this is not for editor-like text replacement repetition.</p>
<p>Note:</p>
<pre>
*1 C89 Rationale 3.8.3 Macro replacement
C99 Rationale 6.10.3 Macro replacement
</pre>
<h3><a name="2.6.2" href="#toc.2.6.2">2.6.2. Argument Expansion Before Substitution</a></h3>
<p>What is essential for achieving the principle of macro expansion parallel to function calls is to expand macros in an argument first before substituting a parameter with the argument. And, the macro call within an argument must have been completed within that argument (it causes an error if not completed.) A macro within an argument must not absorb the text right after it. Thus, nested function-like macro calls can maintain logical clarity. *1</p>
<p>Note:</p>
<p>*1 Refer to <a href="#3.4.25">3.4.25</a>.</p>
<h4><a name="2.6.2.1" href="#toc.2.6.2.1">2.6.2.1. No Expansion for the Operand of the # or ## Operator</a></h4>
<p>Operands for the # operator, however, are not supposed to be macro-expanded. Operands for the ## operator are not macro-expanded, either. The pp-token generated by concatenation becomes subject to macro expansion at rescanning. Why do we need this specification?</p>
<p>This specification is meaningful when an argument includes macros indeed. It is helpful if you want to stringize or concatenate token sequences including macros as they are. On the contrary, to expand a macro before stringizing and concatenation, it needs wrapping another macro which does not use # and ## operators. In order for a programmer to be able to choose either of these, a specification is needed so that no macro-expansion is performed for the operands of # and ## operators. *1</p>
<p>Note:</p>
<p>*1 Refer to <a href="#3.4.25">3.4.25</a>, misc.t/PART 3 and 5. A typical example where the specification of no macro expansion for the operand of the # operator helps is the assert() macro. Refer to <a href="#5.1.2">5.1.2</a>.</p>
<h3><a name="2.6.3" href="#toc.2.6.3">2.6.3. Rescanning</a></h3>
<p>Now, after macro calls are replaced with a replacement list and function-like macro arguments are substituted for parameters in the replacement list after being expanded, the replacement list is rescanned searching for more macro calls.</p>
<p>This rescanning is a specification since K&amp;R 1st. And there seems to have been an "editor-like text replacement repetition" approach in the background. In Standard C, however, function-like macro arguments have been completely expanded already except for operands of ## operators. What on earth is expanded at rescanning?</p>
<p>Rescanning is necessary for macros in replacement list other than parameters, and for macros with ## operators. What is needed rescanning other than those is so-called cascaded macros, where macro "definitions" are nested. If "arguments" for macro calls are nested, they usually do not get expanded again at rescanning since they have been expanded in the nesting structure before rescanning (though there are exceptions. Refer to <a href="#2.7.6">2.7.6</a> and <a href="#3.4.27">3.4.27</a>.)</p>
<h3><a name="2.6.4" href="#toc.2.6.4">2.6.4. Prevention of Recursive Macro Expansion</a></h3>
<p>Although cascaded macros are expanded one after another, it may be a problem sometimes. That is the case the macro definition itself is recursive. If this is expanded as is, it will fall into infinite recursion. The same problems occur in not only the direct recursive case where a macro is included in the definition itself, but also the indirect recursion of 2 or more definitions. In order to avoid this situation, Standard C adds a specification of "If the name of the macro being replaced is found during the rescan of the replacement list, it is not replaced." The phrases are difficult, but its intention is easy to understand.</p>
<p>This is a point where a function-like macro has different grammar from the function. It is also different from the editor-like replacement. Since it is a macro specific specification and has been used as a convenient processing, which is used only in macros, I think it is appropriate to keep this specification by clearly defining it.</p>
<br>
<h2><a name="2.7" href="#toc.2.7">2.7. Issues</a></h2>
<p>I have covered above only good aspects or simple and clear aspects of Standard C preprocessing specifications. If I go into more detail, however, there are parts that are irregular or lacking in their utility or portability for their implementation overhead. Most of these are there without being able to settle traditional or implicit pre-Standard preprocessing methods. The existence of this type of useless area like an appendix which confuses the specification and makes implementation troublesome. Standard C also includes a few parts which caused new unnecessary complications. Those problems are sorted out below.</p>
<h3><a name="2.7.1" href="#toc.2.7.1">2.7.1. Header Name in the &lt;stdio.h&gt; Format</a></h3>
<p>Although header-names enclosed by &lt; and &gt; have been used traditionally since K&amp;R 1st, they are extremely exceptional and irregular as tokens. Because of this, Standard C has many undefined and implementation-defined parts regarding header-name pp-token. For example, it is undefined if the /* sequence is included. Also, if it is not a header-name as in &lt;stdio.h&gt;, the part which is divided into multiple pp-tokens as in &lt;, stdio, . , h, and &gt; must be combined to one pp-token as far as the #include line goes. That method is implementation-defined. Tokenization is performed in translation phase 3. However, if the line turns out to be a #include directive in phase 4, tokenization needs to be redone. I would have to say it is a very illogical specification. The processing in case a space exists between (temporary) pp-tokens, which were once tokenized in phase 3, is also implementation-defined. Directives such as #include &lt;stdio.h&gt; appear to have most portability, but have low portability in respect of preprocessing implementations. Irregularity increases more if the argument of the #include line is a macro.</p>
<p>Header-names enclosed by " and " have no problems like these. However, \ is not handled as an escape character as in header-names enclosed by &lt; and &gt; and this is the difference from string literals. It is not illogical that no escape sequence exists in a header-name which is processed in phase 4 since escape sequences are processed in phase 6 (\ within a header-name is undefined by the Standard. This must be a consideration to ease implementation. In reality, no problem occurs unless \ comes right before " inside of " and ". It is a little more complicated inside of &lt; and &gt;.) *1</p>
<p>Also, the difference between #include &lt;stdio.h&gt; and #include "stdio.h" is only that the former searches a specific implementation-defined location while the latter first searches (a relative path from) the current directory and the same location as &lt;stdio.h&gt; if it is not found (as Standard C does not assume an OS, it does not use the term, "current directory." But it is interpreted for most operating systems.) Therefore, #include &lt;stdio.h&gt; can be simply written as #include "stdio.h".</p>
<p>By having two kinds of header-name formats, there is a readability advantage of spotting at a glance the distinction between user defined and system provided headers. But it does not have to go to the trouble of providing an irregular token for that purpose. All it needs is to distinguish one from another by using different suffixes as in "stdio.h" and "usr.H" (if I add just in case, it is acceptable if a system does not distinguish uppercase and lowercase of a filename since this is a readability issue. Naturally, they can be "usr.hdr", "usr.usr", "usr.u", and etc.)</p>
<p>I believe that header-names enclosed by &lt; and &gt; should be abolished since it serves no use as a language specification and complicates preprocessing tokenization. It cannot be abolished out of the blue, but I would like it to be specified as an obsolescent feature.</p>
<p>Note:</p>
<p>*1 UCN starting with \ was introduced by C99, which is a little troublesome.</p>
<h3><a name="2.7.2" href="#toc.2.7.2">2.7.2. # Operator Specification with Legacy from Character-based Preprocessing</a></h3>
<p>The next problem is the handling of white spaces as token separators between operands of # operators. One or more white spaces are compressed into one space and no space is inserted if there is no white space.</p>
<p>This is half-defined specification. In order to ensure token-based operations, the existence of token separators must not have an influence. For that reason, it should have been defined so that all token separators are deleted or a space is placed between every pp-token. C89 Rationale 3.8.3.2 (C99 Rationale 6.10.3.2) states that the # operator was decided "as a compromise between token-based and character-based preprocessing discipline" within this specification.</p>
<p>This compromise led to an extra burden rather than easing preprocessor implementation and brought ambiguity to complicated macro expansion as well. There is an example shown below in Example 4 of C90 6.8.3, C99 6.10.3 Macro replacement -- Examples.</p>
<pre style="color:navy">
#define str(s) # s
#define xstr(s) str(s)
#define INCFILE(n) vers ## n
#include xstr(INCFILE(2).h)
</pre>
<p>This #include line is expanded as:</p>
<pre style="color:navy">
#include "vers2.h"
</pre>
<p>This example is filled with many problems. There is no vagueness in INCFILE(2) being replaced with vers2. However, the expansion result of INCFILE(2).h, an argument for xstr(), is a sequence of 3 pp-tokens, vers2, ., and h. The expansion example in the Standard is handled with no white spaces among 3 pp-tokens. This involves issues as below.</p>
<ol>
<li>vers2 is not a pp-token which was in source, but generated by macro replacement. To guarantee there are no white spaces after vers2, macro replacement must not generate white spaces before or after. However, pp-tokens may implicitly merge as a result of macro expansion at least for the preprocessors which are independent of the compiler if macro replacement is always such. This is against token-based preprocessing principles.<br>
<br>
<li>Without generating white space before and after the macro replacement, which can be the operands of # operators and to avoid the implicit concatenation of pp-tokens, a little trick is necessary; for the macro replacement which exists in an argument for a function-like macro call, wrap the replacement result with temporary white spaces internally, delete them if it becomes an operand for a # operator, and replace only the temporary white spaces left after all replacement is complete with real spaces (*1.) This is quite a burden for a preprocessing implementation. And there is no merit. Furthermore, it is not clear from the Standard text that this type of processing is necessary and it is unclear what is the right process.<br>
</ol>
<p>All these ambiguity and complexity come from the incompleteness of token separator handling in operands for # operators.</p>
<p>I think it is better to have the specification such that # operators are stringized after each pp-token is separated with a single space in order to avoid implicit concatenation of pp-tokens and causing complicated problems and to show what kind of pp-token sequence the argument stringized is. If defined that way, this macro will be expanded as "vers2 . h". Needless to say, this is not an appropriate macro.</p>
<p>As this example shows, the only case where it will be troublesome to insert a space where none exists is the macro for the #include line using # or ## operator. The #include line to be processed in translation phase 4 cannot use the concatenation of the string literals to be processed in phase 6. However, the macro for the #include line can be simply defined as a string literal without bothering to be parameterized using # and ## operators. Sacrificing token-based principles just because of this parameterization is a great loss in the balance.</p>
<p>In the Standard C preprocessing specification, the syntax is token-based while the semantics specification of # operators is suddenly character-based, losing logical consistency.</p>
<p>Moreover, this example of the Standard assumes the specification, which is not necessarily clear from the Standard text. It is an inappropriate example and should be deleted.</p>
<p>Note:</p>
<p>*1 <b>mcpp</b> was compelled to be implemented in the same way.</p>
<h3><a name="2.7.3" href="#toc.2.7.3">2.7.3. White Space Handling at Macro Re-definition</a></h3>
<p>There is a similar specification to white space handling in operand of # operators with regards macro re-definition. It is defined; "A macro re-definition must be equivalent to the original macro. In order to be equivalent, the number and name of parameters must be the same and the replacement list must have the same spellings. However, in the case of white spaces in the replacement list, their existence must be the same though the number can be different."</p>
<p>If the specification of the # operators are as above, this is an obvious conclusion since same handling is necessary for white spaces in the replacement list. Still, the cause of the problem is the specification of the # operators.</p>
<p>If the # operator is handled in such a way that one space exists between every pp-token in operands, there will be no issue regarding the existence of white spaces for macro re-definition.</p>
<p>Moreover, this can be generalized in the preprocessor implementation, by replacing with one space between every pp-token in source as a principle. By doing so, tokenization for macro expansion can be done easily and accurately. However, there are two exceptions to this principle. One is &lt;newline&gt; in the preprocessing directive line and another is whether there are white spaces between a macro name and the subsequent '(' in macro definition. This traditionally has been the basis of preprocessing in C and cannot be changed after all these years.</p>
<h3><a name="2.7.4" href="#toc.2.7.4">2.7.4. Parameter Name at Function-like Macro Re-definition</a></h3>
<p>I have mentioned in 2.7.3 that parameter names must match regarding macro re-definition in the specification, but I believe this is an excessive specification. Parameter names, of course, do not make any difference to macro expansion. However, in order to check for re-definition, a preprocessor needs to store parameter names of all macro definitions. Even so, its usage is nothing other than re-definition checking within the specification. It is not such a great idea to give overhead to implementations only for almost meaningless checking.</p>
<p>I think it is better to remove the specification that parameter names must match at macro re-definition.</p>
<h3><a name="2.7.5" href="#toc.2.7.5">2.7.5. Unpredictable Evaluation of Character Constant in #if Expression</a></h3>
<p>#if expression as an argument for the #if line is a constant expression in the integer type. Its evaluation must be independent of the execution environment since it is done in preprocessing. Because of that, a cast, the sizeof operator, and enum constants, which require references to the execution environment (these are first evaluated in translation phase 7) are excluded from #if expression compared to standard integer constant expressions. Character constants (and wide character constants), however, are not excluded.</p>
<p>Character constant evaluation is implementation-defined with many factors as shown below and has little portability.</p>
<ol>
<li>Even the value of a basic character differs depending on the basic character set (ASCII, EBCDIC, and others.)<br>
<br>
<li>Even a single-character character constant, within the same basic character set, has implementation-defined sign handling (depending on whether char is signed in the compiler proper.)<br>
<br>
<li>Multi-character character constant evaluation is implementation-defined and the value may not be the same even if the sign handling is the same as basic character set. It is not defined whether 'ab' is 'a' * 256 + 'b' or 'a' + 'b' * 256 when <tt>CHAR_BIT</tt> is 8 and char is unsigned.<br>
<br>
<li>Multi-byte character encoding is implementation-defined. Wide character encoding is same as multi-byte character encoding. The size of wchar_t and whether it is signed or unsigned is implementation-defined.<br>
<br>
<li>Even if multi-byte character encoding is the same, the evaluation of a character may not. There is an issue same as 3.<br>
<br>
<li>All above are common problems with character constant evaluation in compiler proper. In addition, the character set in preprocessing can differ from the one the compiler proper sees.<br>
A source character set is applicable up to translation phase 4 and an execution character set is applicable at phase 6 and after. Phase 5 converts the characters in character constants and string literals from the source character set into the execution character set. That is to say, either or both basic character sets and multi-byte character encodings also may differ between source and runtime.<br>
Character constants in the #if expression are evaluated in phase 4. This can be either the value of the source character set or simulated value of the execution character set. It is not defined as a source character set either.<br>
<br>
<li>Even if the character set evaluated in a #if and the execution character set are the same, the methods of evaluation can differ. That is to say, sign handling and the byte order of multi-character character constant and multi-byte character constant evaluation may differ between phase 4 and phase 7.<br>
<br>
<li>Furthermore, while the character constants including multi-byte character constants are evaluated in int and wide character constants in wchar_t in phase 7, they are all evaluated in long or unsigned long in phase 4. In other words, int is handled as if it has the same internal representation as long and unsigned int as if it has the same internal representation as unsigned long in phase 4. Therefore, for the implementation of <tt>INT_MAX</tt> &lt; <tt>LONG_MAX</tt>, even if character sets, sign handling, evaluation byte orders are totally identical between phase 4 and 7, the character constant which does not overflow in phase 4 can overflow in phase 7. As a negative number in phase 7 may be a long positive number in phase 4, even whether it is positive or negative is not always the same. The integer constant token, which is not a character constant, does not become a negative number, however, whether positive or negative can hardly be predicted in general for character constants.<br>
<br>
<li>In C99, the #if expression type became the maximum integer type of its implementation. In other words, the evaluation type may vary depending on the implementation.<br>
<br>
<li>In addition, multi-byte characters have an encoding problem though it exists in compilation as well as preprocessing. For example, UTF-8 encodes a two byte Unicode character in between one byte and three bytes, however, what is the "value" of its character constant? Is a Kanji value the "value" which results from evaluating the three byte UTF-8 code or the "value" of the original Unicode? It will be implementation-defined, however, it is not even clear what sort of specification is reasonable.<br>
<br>
<li>Though this is also a common problem with compilation, UCN was also introduced in C99 and C++98. Is a character expressed in UCN and one written in multi-byte characters the same "character"? They should be the same in nature, however, their "value" will be different depending on the encoding of multi-byte characters.<br>
</ol>
<p>As above, little can be predicted how evaluation is done since the character constant values for the #if expressions have no portability among implementations and may differ even within the same implementation depending on the compilation phases.</p>
<p>In general, the specification of the C language integer type has few ambiguous parts. Although negative value handling is implementation-defined with respect to computation, they are CPU-dependent and there are few parts implementers can decide optionally with character constant evaluation as an only exception. Many aspects are determined at the discretion of implementers other than the CPU specification, basic character set, and multi-byte character encoding.</p>
<p>The range of discretion for implementations increases immensely for the character constants of the #if expressions and no matching is guaranteed among compilation phases. Little of what is evaluated is understood even if this is evaluated. Character constant evaluation can be thought to require a reference to the execution environment under normal conditions. Standard C preprocessing removed this process of requiring a reference, but not character constants only somehow. And, it seems that the specification, which does not require a reference, was forced to be created, which created ambiguity.</p>
<p>How are this type of character constants for #if expressions used? The value of a char type variable is often compared with a character constant in the compilation phase, but there is no usage in the preprocessing phase in which no variables are used. I cannot think of any appropriate examples for the use of a character constant in a #if expression. This is a useless thing and should be removed from the #if expression subject just as cast and sizeof. There will be less source with issues if this is removed compared even with cast or sizeof removal.</p>
<h3><a name="2.7.6" href="#toc.2.7.6">2.7.6. Non-Function-like Rescanning of Function-like Macro</a></h3>
<p>Macro calls are once replaced with the replacement list, and then rescanned. The messiest thing in the rescanning specification in Standard C is that the token sequence after the macro call is rescanned continuously to the replacement list as if the sequence is following the replacement list. This is completely deviated from the principle of function-like macro specification modeled after function calls and becomes the most outstanding factor to make macro expansion incomprehensible. I think that this specification of the subsequent token sequence as a rescanning subject should be removed and that the rescanning subject should be limited to the replacement list only.</p>
<p>Actually, rescanning subsequent token sequence seems to have been a long time implicit specification since around K&amp;R 1st. This specification is no longer necessary in Standard C, but remained in an appendix as a legacy. Since this issue concerns the basis of macro expansion, I will study it further in detail below.</p>
<p>It is not an easy thing to describe the macro rescanning method in writing. The text in the Standard or in K&amp;R 2nd is not easy to understand, either. For example, K&amp;R 2nd A.12 says "the replacement list is repeatedly rescanned." However, the Standard does not state "repeatedly". It can be read as one rescanning. It can also be read as recursive rescanning, but not clearly described as such either.</p>
<p>This cannot be explained accurately without using an actual example. Furthermore, it cannot be understood intuitively without explaining the implementation method. That is how close macro rescanning is to the traditional implementation of macro expansion.</p>
<p>First, we will take a silly example. To simplify the problem, assume x and y are not a macro. How can this macro call be expanded?</p>
<pre style="color:navy">
#define FUNC1( a, b) ((a) + (b))
#define FUNC2( a, b) FUNC1 OP_LPA a OP_CMA b OP_RPA
#define OP_LPA (
#define OP_RPA )
#define OP_CMA ,
FUNC2( x, y)
1: FUNC2( x, y)
2: FUNC1 OP_LPA x OP_CMA y OP_RPA
3: FUNC1 ( x , y )
</pre>
<p>It becomes clear at once that 1: is replaced with 2: and 3: is generated by rescanning in 2:. Then, is 3: a macro call? More specifically, should this be rescanned again from the beginning?</p>
<p>Is rescanning something repeated many times from the beginning or whose applicable range is gradually narrowed down recursively? The truth is neither.</p>
<p>As a matter of fact, rescanning seems to have been performed in a certain type of exceptional recursion or in a certain type of repetition resulting in the same. Its classical example is the one in the Macro Processing chapter in "Software Tools" by Kernighan &amp; Plauger. This is something to be developed into the M4 macro processor later and this itself is not a C preprocessor. It is indicated that the macro processor was originally designed and implemented in C by Ritchie. The prototype of the preprocessor implementation is available.</p>
<p>In this macro processor, rescanning is realized by sending back the replacement list to the input for re-reading when there is a macro call. When there is another macro call in the replacement list, the new macro replacement list is sent back and re-read as if "it had been in the input originally." As it is written; "it provides an elegant way to implement the rescanning of macro replacement text", "This is how we handle the recursion implicit in nested sources of input", and others, this method greatly helps macro processor program to be structured and understood easily.</p>
<p>Many of C preprocessors perform rescanning by putting the replacement list in pseudo inputs, a type of stack, and re-reading it.</p>
<p>In the example above, if FUNC1 turns out not to be a macro call at this point when 2: is being rescanned, this token is established at the same time and the replacement hereafter will be for OP_LPA and after whether repetition or recursion. If OP_LPA is replaced with ( and turns out not to be a macro, x and later will be applicable next. This way, establishing a token is done sequentially starting with the beginning and 3: becomes the last result. This is no longer a macro call.</p>
<p>This method since "Software Tools" (or even before that) is certainly a concise implementation method. Though not mentioned in "Software Tools", there is also a pitfall. The problem is that there is a chance that rescanning may scan the part after the original macro call beyond the replacement list since the replacement list sent back to the input is read-in consecutively with source. In nested macros, the nesting level may get shifted; unnoticed while rescanning. A macro without arguments expanded into the name of the macro with arguments and a abnormal macro where the replacement list comprises the first half of another macro call with arguments causes this situation.</p>
<pre style="color:navy">
#define add( x, y) ((x) + (y))
#define head add(
head a, b)
</pre>
<p>This is the example. This strange macro call is expanded as ((a) + (b)). This for some reasons ended up with being officially acknowledged by Standard C. In fact, this macro is legal rather than undefined.</p>
<p>I cannot think that C preprocessors were intended for abnormal macros like this. However, I wonder perhaps whether the original C preprocessor implementation was as above, which resulted in expanding these macros somehow in silence, and some programs consciously took advantage of these holes to the point where this became a de facto standard specification and finally approved in Standard C. That is to say, a small defect in the original C preprocessor implementation led to a strange de facto standard and left a trail even in Standard C. This is the reason an appendix is an appendix.</p>
<p>Now, returning to the topic of whether rescanning is recursive or repetitive, I believe that it is not necessarily wrong to say this is either an irregular recursion or repetition. It is recursion, but it has the strange characteristic that it is not always narrowed down its applicable range as in ordinary recursion, but rather the range is gradually shifted. This is repetition. However, the repetition is not from the beginning, but from the middle by including following parts gradually.</p>
<p>Therefore, it is possible to process the text, after all comments and preprocessor directives are processed, from the beginning until the end with this shifted rescanning only. In fact, such method is used in "Software Tools" and there is some using a similar way in the current C preprocessor source. In other words, rescanning is a synonymous with macro expansion and also the macro expansion for all text.</p>
<p>The fact that rescanning subjects are shifted gradually causes many problems. The next example was listed in C89 Rationale 3.8.3.4 (C99 Rationale 6.10.3.4) as an example of a macro, which is unclear how to be expanded. It is stated that the reason why this process was not defined as a specification was that "as the Committee saw no useful purpose in specifying all the quirks of preprocessing for such questionably useful constructs." However, this example is suggestive. Rather, this was not possible to be defined as a specification.</p>
<pre style="color:navy">
#define f(a) a*g
#define g(a) f(a)
f(2)(9)
</pre>
<p>In this example, f(2) is replaced with 2*g at first. If the "subsequent preprocessing tokens" is not to be rescanned, macro expansion is completed and f(2)(9) becomes the token sequence of 2*g(9). However, as the "subsequent token sequence" is applicable, this g(9) forms a macro call and replaced with f(9). Here, it is not clear whether this f(9) should be replaced with 9*g again or not by applying the rule of no re-replacement for the macro with a same name. The token sequence of f(9) is generated by rescanning the continuation of g, that is the end of the first replacement result of f(2), and (9) of the "subsequent token sequence" and it is unclear whether this is inside or outside the f(2) call nest.</p>
<p>This problem was corrected in C90 Corrigendum 1, which adds the next example to Annex G.2 Undefined behavior.</p>
<blockquote>-- A fully expanded macro replacement list contains a function-like macro name as its last preprocessing token (6.8.3).</blockquote>
<p>This correction, however, only causes more confusion.</p>
<p>First of all, the wording, "fully expanded macro replacement list", is not clear in meaning. This can be only interpreted as "the replacement list after the macro within the argument is expanded if there is an argument." In that case, in the example of f(2)(9), f(2) is replaced with 2*g before considering the re-replacement of the macro with a same name and it becomes undefined already when g is a function-like macro name by rescanning it. In other words, if f is called, it always becomes undefined in this f and g macro definition.</p>
<p>If this "correction" is applied, the following example for macro rescanning in ISO/IEC 9899:1990 6.8.3 Examples will be undefined to begin with.</p>
<pre style="color:navy">
#define f(a) f(x * (a))
#define x 2
#define g f
#define w 0,1
#define t(a) a
t(t(g)(0) + t)(1); /* f(2 * (0)) + t(1); */
g(x+(3,4)-w) /* f(2 * (2+(3,4)-0,1)) */
</pre>
<p>The Standard states that these macro calls will be expanded as in the comments, but this is not the case if the Corrigendum is applied. In these macro definitions of f and g, they will be always undefined when the g identifier appears. It is because f, a function-like macro name, is the only and last pp-token in the replacement list for g.</p>
<pre style="color:navy">
t(t(g)(0) + t)(1)
</pre>
<p>At first, the argument of the first t call will be expanded.</p>
<pre style="color:navy">
t(g)(0) + t
</pre>
<p>Since there is another macro call, t(g), it will be expanded, but the argument must be expanded first for that.</p>
<pre style="color:navy">
g
</pre>
<p>And if this is replaced with f, it will become undefined here.</p>
<p>Even if replacements are continued as is, it will become:</p>
<pre style="color:navy">
t(f)
f
</pre>
<p>And it will be undefined again since the last pp-token of the t(f) expansion result is f. If replacements continue further, it will become:</p>
<pre style="color:navy">
f(0) + t
f(x * (0))
f(2 * (0))
f(2 * (0)) + t
t(f(2 * (0)) + t)
f(2 * (0)) + t
</pre>
<p>This ends the expansion of the first t call in any case, but it will be undefined for the third time since the end of this replacement list is a function-like macro name, t.</p>
<p>How about the following?</p>
<pre style="color:navy">
g(x+(3,4)-w)
</pre>
<p>This will be undefined by the time g is replaced with f.</p>
<p>This results in confusion by contradicting Examples with G.2.</p>
<p>If the examples in the Examples are omitted, the correction in the Corrigendum does not relieve confusion. First of all, G.2 is not a part of the Standard and this addition does not have grounds in the text of Standard proper. In the text of Standard, it is only written that the "subsequent token sequence" is also to be rescanned. Secondly, even if this Corrigendum is included in the Standard proper,</p>
<pre style="color:navy">
#define head add(
</pre>
<p>in the previous example, add is correct since it is not in the end of the replacement list.</p>
<pre style="color:navy">
#define head add
</pre>
<p>is undefined, however. This is too unbalanced. Also, there is an issue of the wording, "fully expanded", being unclear in meaning. *1, *2</p>
<p>It goes without saying that these are quirks brought by the specification on "subsequent token sequence" as rescanning subject. The more plausible they try to make it sound, the more confusing it gets. The Standard states the specification forbidding the replacement of the macro with a same name in extremely difficult sentences. A reason for this difficulty comes also from these quirks.</p>
<p>On the other hand, Standard C defines that macro expansion in an argument must be performed only within the argument for function-like macro calls. Since it will be turmoil if the macro expansion in an argument eats up the text behind it, this is no wonder.</p>
<p>As a result of this, however, an imbalance occurs between the macro within an argument and not so.</p>
<pre style="color:navy">
#define add( x, y) ((x) + (y))
#define head add(
#define quirk1( w, x, y) w x, y)
#define quirk2( x, y) head x, y)
head a, b);
quirk1( head, a, b);
quirk2( a, b);
</pre>
<p>In this quirk1() call, it will be a violation of constraint as an incomplete macro call at rescanning after the first argument, head, is replaced with add(. Put simply, it is an error. However, quirk2() and head a, b) will not be an error, but expanded as:</p>
<pre style="color:navy">
((a) + (b))
</pre>
<p>It may sound repetitious, but this type of absurdity all comes from the fact that even the "subsequent token sequence" is applicable for macro rescanning in general. As a matter of implementation, the nesting level information needs to be added in order for the argument expansion to be performed independent of other text parts even using the method of sending the replacement list back to input. By using that method, it will be easy not to have the "subsequent token sequence" rescanned in general. Rather, in the current half-baked specification, it is necessary to change the process depending on whether it is inside an argument or not, resulting in extra load for implementations.</p>
<p>Macro expansion in C has been traditionally influenced by editor-like string replacement. We can say that pre-Standard macro expansion is something that has been added to string replacement for editors and become complicated to excess.</p>
<p>By contrast, Standard C took the trouble to name the macro with arguments a function-like macro. I can guess that it tried to bring the call syntax closer to a function call. The specification that the macro in an argument is replaced with a parameter after being fully expanded and the one that the expansion is performed only within the argument conform this principle. However, this principle is spoiled by the specification that macro rescanning in general includes the subsequent token sequence. It is an inheritance of text replacement repetition from its ancestor.</p>
<p>If the subsequent token sequence was removed from the rescanning subject, it could have been defined that macro expansion is completely recursive, and that the applicable range is narrowed or (at least not extended) forward or backward on every recursion. And, it would have been clear as an appropriate macro for the name, function-like macro. I cannot think there is much source code which would have had problems by this decision. I can only think that ANSI C committee could not make a decision on cutting an appendix inherited from an ancestor. *3</p>
<p>I wished C99 would cut it off cleanly, but the appendix has survived again.</p>
<p>Note:</p>
<p>*1 The object-like macro which is expanded into a function-like macro name is sometimes seen in actual programs. It is as below.</p>
<pre style="color:navy">
#define add( x, y) ((x) + (y))
#define sub( x, y) ((x) - (y))
#define OP add
OP( x, y);
</pre>
<p>This is not as abnormal as an expansion into the first half of the function-like macro call as the former, but there is no reason why it must be this way. It is good to define a function-like macro nesting in a function-like macro as below.</p>
<pre style="color:navy">
#define OP( x, y) add( x, y)
</pre>
<p>*2 The reason for this correction by Corrigendum is in "Record of Responses to Defect Reports" by C90 ISO C committee (SC 22 / WG 14) (#017 / Question 19.) The question on the macro expansion in f(2)(9) of ANSI C Rationale 3.8.3.4 was brought up again. The direct issue in this example must have been the application range of the specification on "prohibiting the re-replacement of macros with a same name", but the committee has answered as a common problem unlimited to macros with a same name. They did not realize that this interpretation might cause contradictions in the Examples.</p>
<p>In addition, this wording, "fully expanded", is strange. When f(2) is replaced with 2*g and rescanned up to g, is it fully expanded? If so, no more replacements will be performed. Therefore, it will not be undefined, either. If not fully expanded yet, g is rescanned with the succeeding (9) and replaced by f(9). If this is fully expanded, the last pp-token of 2*f(9) is not a function-like macro name. Therefore, this answer does not apply. In other words, it says "after macro expansion is completed" where the issue is when macro expansion ends. Thus, when macro expansion ends became more confusing.</p>
<p>In C99 draft in November, 1997, this item in the Corrigendum was included in Annex K.2 Undefined behavior but deleted in the draft in August, 1998 replaced by a following paragraph below in Annex J.1 Unspecified behavior. This eventually was adopted in C99.</p>
<blockquote>
<p>When a fully expanded macro replacement list contains a function- like macro name as its last preprocessing token and the next preprocessing token from the source file is a (, and the fully expanded replacement of that macro ends with the name of the first macro and the next preprocessing token from the source file is again a (, whether that is considered a nested replacement.</p>
</blockquote>
<p>It seems that the committee finally realized the contradiction in the Corrigendum. Fundamental problems in the text proper of the Standard, however, still remain. Also, when macro expansion ends is unspecified in the end. Furthermore, the distinction by the presence of '(' in the source means the difference in the result when the same macro exists in the source and when it exists in the replacement list of another macro. This is an inconsistent specification.</p>
<p>On this issue, refer also to section <a href="#3.4.26">3.4.26</a>.</p>
<p>Also, the specification of macro expansion in C++ Standard is same as C90 without an equivalent of Corrigendum 1 in C90 nor the specification added in Annex J.1 of C99.</p>
<p>*3 Even with this decision, FUNC2( x, y) in the previous example will be FUNC1 (x, y) in argument expansion if this is in the argument of another macro call and again expanded into ((x) + (y)) at the original macro rescanning. In other words, the final expansion results differ depending whether in an argument or not so. However, this is another level of problem and not an inconvenience.</p>
<h3><a name="2.7.7" href="#toc.2.7.7">2.7.7. C90 Corrigendum 1, 2 and Amendment 1</a></h3>
<p>With respect to ISO/IEC 9899:1990, Corrigendum 1 was released in 1994, Amendment 1 in 1995, and finally Corrigendum 2 in 1996.</p>
<p>Corrigendum 1 contains trivial corrections in wording mostly but only 2 impacts preprocessing. One is regarding macro rescanning of 2.7.6 described above.</p>
<p>Another is a specification extremely special regarding the case that a macro name in macro definition includes $ and others.</p>
<p>In Standard C, '$' is not accepted as a character in an identifier though there are implementations allowing this traditionally. In an example of 18.9 in test-t/e_18_4.t, $ is a character and interpreted as a pp-token in Standard C. The macro name is THIS and $ and after becomes the replacement list of an object-like macro, which is totally different result from the intention of the program which is a function-like macro with the name, THIS$AND$THAT.</p>
<p>In Corrigendum 1, an exception specification was added regarding this type of example; "if object-like macro replacement list starts with a non-basic character, a macro name and a replacement list must be separated by white-space." Standard C must output a diagnostic message to this example in 18.9. It is supposed to preventing the situation where the source with $ or @ used in macro names is silently preprocessed into an unintended result. It is a painstaking specification, but it is annoying that this type of exception increases. In implementations not accepting $ and/or @ as an identifier, macros like this always become an error in the compilation phase even if they are not an error in preprocessing. So, it does not seem to be necessary to define this exception specification. *1</p>
<p>In addition, ISO 9899:1990 had an ambiguous constraint that a pp-token, header-name, can be appear only in #include directives. However, it was corrected in Corrigendum 1 so that a header-name is recognized only in a #include directive.</p>
<p>In Amendment 1, the core is multi-byte characters, wide characters, and the library functions operating those strings. Accompanied by those, &lt;wchar.h&gt; and &lt;wctype.h&gt; standard headers were added. In addition, the &lt;iso646.h&gt; standard header and the specification on digraphs are added as alternatives to trigraphs as characters not included in ISO 646 character set or a notation method for token and pp-token using those characters. &lt;iso646.h&gt; is a quite easy header which define some operators as macros and it does not have any special problems. *2</p>
<p>The problem is a digraph. This is very similar to a trigraph and the usage is almost the same though the positioning in preprocessing is completely different. A trigraph is a character and converted into a regular character in translation phase 1 while a digraph is a token and pp-token. If a digraph sequence is stringized by the # operator, it must be stringized as is without conversion (though this # itself is also written as %: in a digraph.) Because of this, and only this, implementations need to retain this as a pp-token at least until phase 4 completes. If it were to be converted, it is later (convert the digraph sequence left as a token, not in string literal.)</p>
<p>This imposes an unnecessary burden on implementations. Implementations are more concise recognizing a digraph as a character just as a trigraph and to convert it in phase 1. There is no benefit to keep this as a pp-token. The Amendment also notes that the difference between a digraph and a usual token occurs only when they are stringized. It might be seen that it would be troublesome in writing a string literal like "%:" if a conversion is done in phase 1. But this is similar in trigraphs and too special of a problem to consider. If it must be written, just "%" ":" is enough. Digraphs should be re-positioned so that they are converted in phase 1 as an alternative to trigraphs.</p>
<p>Corrigendum 2 has no corrections regarding preprocessing.</p>
<p>Note:</p>
<p>*1 This specification disappeared in C99 and C++ Standard.</p>
<p>In C99, the following generalized specification was added to 6.10.3 Macro replacement/Constraints instead.</p>
<blockquote>There shall be white space between the identifier and the replacement list in the definition of an object-like macro.</blockquote>
<p>As this is also a tokenization exception specification, it is not praiseworthy.</p>
<p>*3 In C++ Standard, these identifier-like operators are tokens, not macros. Though it is difficult to understand why it is so (could it be an idea to cut down what preprocessing must do?), it is troublesome for implementations in any case.</p>
<h3><a name="2.7.8" href="#toc.2.7.8">2.7.8. Redundant Specifications</a></h3>
<p>In C90 5.1.1.2, C99 5.1.1.2 Translation phases 3, there are redundant specifications though they are harmless.</p>
<blockquote>A source file shall not end in a partial preprocessing token or comment.</blockquote>
<p>Since translation phase 2 specifies that source files must not end without a &lt;new-line&gt; or with &lt;backslash&gt;&lt;newline&gt;, the source file that passes phase 2 always end with a &lt;newline&gt; without a &lt;backslash&gt;. It never ends with a partial preprocessing token. Within Partial preprocessing token categories, there are ", ', &lt;, and &gt;, which are unmatched in logical lines, which are considered to be undefined in C90 6.1 Lexical Elements/Semantics and not problems limited to the source ending. "Partial preprocessing token or" is unnecessary wording.</p>
<p>C90 6.8.1, C99 6.10.1 Conditional inclusion/Constraints contains expressions, which cause a misunderstanding.</p>
<blockquote>it shall not contain a cast; identifiers (including those lexically identical to keywords) are interpreted as described below;</blockquote>
<p>This "it shall not contain a cast; " is superfluous. In the succeeding parts and Semantics, it is made clear that all identifiers including an identifier the same as a keyword are expanded if macro and remaining identifiers are evaluated as 0. A cast does not need to be considered. In the (type) syntax, it is clear that type is handled as a simple identifier.</p>
<p>On the contrary, if this is in a constraint, it can be interpreted that the implementation recognizes the cast syntax and must output a diagnostic message. That is not the intention of the Standard. There is no keyword in translation phase 4, cast has no way to be recognized. As far as this is concerned, sizeof is also the same. It is strange that only cast is mentioned without mentioning sizeof. This type of wording is called "superfluous."</p>
<br>
<h2><a name="2.8" href="#toc.2.8">2.8. Preprocessing Specification in C99</a></h2>
<p>The following specification regarding preprocessing was added to C99.</p>
<ol>
<li>The hexadecimal sequence in the \uxxxx or \Uxxxxxxxx format in identifiers, string literals, character constants, or pp-numbers is called UCN (universal-character-name) and means a Unicode character value. This must specify extended characters not included in the basic source character set. Whether a \ should be inserted when a UCN is stringized by the # operator is implementation-defined.<br>
<br>
<li>Implementation-defined characters can be used in identifiers. Therefore, implementations that allow the use of multi-byte-characters such as Kanji characters in identifiers became possible.<br>
<br>
<li>Handle // to the end of the line as a comment.<br>
<br>
<li>As e+, E+, e-, and E-, the sequence of p+, P+, p-, and P- is accepted in a pp-number. This is for writing the bit pattern of a floating-point number hexadecimal such as 0x1.FFFFFEp+128.<br>
<br>
<li>The type of the #if expression is a maximum integer type in the implementation. As long long/unsigned long long is required, the type of the #if expression has the size of long long or wider.<br>
<br>
<li>Variable argument macros can be used.<br>
<br>
<li>An empty argument of a macro call is a valid argument.<br>
<br>
<li>Add a predefined macro, <tt>__STDC_HOSTED__</tt>. This is defined as 1 on a hosted implementation, 0 otherwise. A predefined macro, <tt>__STDC_VERSION__</tt>, is defined in 199901L.<br>
<br>
<li>Add predefined macros, <tt>__STDC_ISO_10646__</tt>, <tt>__STDC_IEC_559__</tt>, and <tt>__STDC_IEC_559_COMPLEX__</tt> as options.<br>
<br>
<li>The new _Pragma operator.<br>
<br>
<li>Reserve the directive name starting with #pragma STDC for the Standard and implementation and add three #pragma STDC directives which show floating point operation methods. Directives starting with #pragma STDC are not to be macro-expanded, but other #pragma lines not so are implementation-defined.<br>
<br>
<li>When a wide-character-string-literal and a character-string-literal are side by side, it was considered to be undefined in C90. However, they are concatenated as a wide character string literal.<br>
<br>
<li>Extend the range of the line number used as an argument for #line to [1,2147483647].<br>
<br>
<li>Raise translation limits as below.<br>
<blockquote>
<table>
<tr><th>Length of a source logical line </th><td>4095 bytes</td></tr>
<tr><th>Length of a string literal, character constant, and header name</th><td>4095 bytes</td></tr>
<tr><th>Length of an internal identifier </th><td>63 characters</td></tr>
<tr><th>Number of #include nesting </th><td>15 levels</td></tr>
<tr><th>Number of #if, #ifdef, #ifndef nesting </th><td>63 levels</td></tr>
<tr><th>Number of parenthesis nesting in an expression </th><td>63 levels</td></tr>
<tr><th>Number of parameters of a macro </th><td>127</td></tr>
<tr><th>Number of macros definable </th><td>4095</td></tr>
</table>
</blockquote>
<li>Header name was guaranteed up to 6 characters + . + 1 character. This is changed to 8 characters + . + 1 character.<br>
</ol>
<p>Variable argument macros are as below. If there is a macro definition,</p>
<pre style="color:navy">
#define debug(...) fprintf(stderr, __VA_ARGS__)
</pre>
<p>a macro call,</p>
<pre style="color:navy">
debug( "X = %d\n", x);
</pre>
<p>is expanded as:</p>
<pre style="color:navy">
fprintf(stderr, "X = %d\n", x);
</pre>
<p>In other words, ... in the parameter list means one or more parameters and __VA_ARGS__ in the replacement list corresponds to it. Even if there are multiple arguments which correspond to ... in a macro call, the result of merging those including ',' is handled as one argument.</p>
<p>Among undefined behaviors in C90, there are some in which adequately meaningful interpretations are possible. An empty argument in macro calls is one of them and there is a case that it is useful to interpret this as 0 pp-token. This became a valid argument in C99.</p>
<p>C99 mentions an extension operator called _Pragma which is converted into #pragma foo bar if written as _Pragma( "foo bar"). In C90, the argument for the #pragma line does not get macro-expanded and the line similar to a #pragma directive as a result of macro expansion is not handled as a directive and cannot write #pragma in the replacement list of macro definition. On the other hand, the _Pragma expression can be written in the macro replacement list and #pragma which came from its result is handled as a directive. The extension by _Pragma tries to improve the portability of cumbersome #pragma.</p>
<p>It is simpler to make a modification that the argument of #pragma is subject to macro expansion, without this type of irregular extension. It will largely achieve the intention of portability improvement. However, in that case, there still remains a constraint that #pragma cannot be written in a macro and there will be an issue that the argument of #pragma which must not be macro-expanded has to have its name changed to start with __ in order to separate from the user name space. Though _Pragma() operator is irregular, its implementation is not so troublesome and it is a reasonable specification.</p>
<p>There are too many issues on the introduction of Unicode. First of all, implementations must prepare a huge table for multi-byte characters and Unicode conversion, causing large overheads. It is virtually impossible to implement it on the systems with 16 bits and less. There are systems that do not handle Unicode. In addition, there are many cases that a Unicode and a multi-byte character do not have a one-on-one mapping. It seems too aggressive to place Unicode in a C language standard in the name of programming language internationalization.</p>
<p>In C99, UCN handling drastically reduced compared with the draft in November, 1997 and C++ Standard. The preprocessing load became small relatively. Therefore, a certain implementation became possible in <b>mcpp</b> as well. *1</p>
<p>However, there are still some large loads on compiler proper. Also, since these are unreadable expressions, I expect that they will end up not being used much as trigraphs. *2</p>
<p>Note:</p>
<p>*1 In the draft in November, 1997, almost same as C++ Standard, it was supposed to be that the extended characters which are not in the basic source character set are all converted into UCN in translation phase 1 and converted again into the execution character set at phase 5.</p>
<p>In case of implementing this, it is speculated that a tool will be called to convert these before and after processing. As the conversion is OS-dependent, separate tools will be realistic.</p>
<p>*2 According to C99 Rationale 5.2.1 Character sets, this specification assumes that unreadable expressions are converted between the source in multi-byte characters by a tool included in the compiler system to be used. This must mean separating multi-byte character string literal parts in a separate file to process. I wonder how practical that is.</p>
<br>
<h2><a name="2.9" href="#toc.2.9">2.9. Toward Clear Preprocessing Specifications</a></h2>
<p>Problems in Standard C preprocessing specifications and what I think mentioned above are also requests for Standard C in the future. In summary, there are following items.</p>
<ol>
<li>Header-names in the &lt;stdio.h&gt; format should be an obsolescent feature. In the next version after the next, header-names should be only in the string literal format.<br>
<br>
<li>Stick to the token-based preprocessing principle. The # operator should be stringized after a single space is inserted between each pp-token even without a token separator so that whether a token separator exists in an argument does not influence.<br>
<br>
<li>Similarly, macro re-definition is not influenced by whether a token separator exists in the replacement list.<br>
<br>
<li>Parameter name differences should not be an issue at macro re-definition, since checking parameter name differences just increases the overheads in implementation and has no value.<br>
<br>
<li>Character constant evaluation normally requires a reference to the execution environment and has no use in the #if expression. Therefore, this should be removed from the #if expression subject.<br>
<br>
<li>Function-like handling of function-like macros should be consistent. Whether a macro call is in an argument or replacement list, macro rescanning should be applicable only for the replacement list and not for the succeeding pp-token sequence of a macro call so that the same pp-token sequence is generated in principle.<br>
<br>
<li>A digraph should not be a token, but an alternate spelling of a character similar to a trigraph. It should be converted in translation phase 1.<br>
<br>
<li>Remove "partial preprocessing token or" from translation phase 3.<br>
<br>
<li>Remove a description regarding the #if expression, "it shall not contain a cast; ", from the constraint and move it to footnote 140 in C99.<br>
<br>
<li>Trigraphs should be abolished as they are not used in Europe.<br>
</ol>
<p>These are all intended to reorganize irregular rules and make preprocessing specifications simple and clear. There is no doubt that these will make preprocessing easier to understand. On the contrary, there should be little annoyance.</p>
<p>I believe that <b>mcpp</b> V.2 implemented all preprocessing specifications in Standard C in the Standard mode including the parts I do not think highly of. In the 'post-Standard' mode, preprocessing with modifications above is implemented (also excluding UCN, the use of multi-byte characters in an identifier.)</p>
<p>Amendment 1 and Corrigendum 1 in C90 took the direction of increasing the irregularity of preprocessing rather than cleaning it up.</p>
<p>C99 added various new features, but did not clean up the confusion of logic above either, unfortunately. *1</p>
<p>Regarding the specifications added in C99, I have the following request.</p>
<ol>
<li>The introduction of Unicode (UCN) should be limited to an option.
</ol>
<p>In addition, there is an issue other than the problems described above regarding evaluation rules for the integer type applicable to #if expressions.</p>
<ol>
<li>There is a constraint that the constant expression shall evaluate to a constant in the range of representable values for its type. It is not clear if this applies to all constant expressions. Since there are no exceptions described, I can only interpret that all constant expressions are applicable, but I expect that the intention of Standard C is something like "where a constant expression is necessary." It should be clear. On the other hand, there is a specification, "computation involving unsigned operands can never overflow." It is vague as to whether a diagnostic message should be output in case that the result of unsigned type constant evaluation goes beyond its range (since a constant expression can be evaluated at compilation, it seems appropriate to output a diagnostic message.)
</ol>
<p>Since this is not a preprocessing specific issue, it will not be discussed further.</p>
<p>Also, in C90 it was defined that the result of / or % when one or both operands are negative is implementation-defined, which was a terrible specification. This became the same specification as div() and ldiv() in C99.</p>
<p>Note:</p>
<p>*1 Various defect reports regarding C99, responses to them, and corrigendum drafts are on the ftp site below. This is the official ftp server for ISO/SC22/WG14 and you can ftp as anonymous at least for now (SC stands for a steering committee and WG means a working group. SC22 deliberates programming language Standards and WG14 handles C Standards.)</p>
<blockquote>
<p><a href="http://www.open-std.org/jtc1/sc22/wg14/">http://www.open-std.org/jtc1/sc22/wg14/</a></p>
</blockquote>
<br>
<h1><a name="3" href="#toc.3">3. Validation Suite Explanation</a></h1>
<h2><a name="3.1" href="#toc.3.1">3.1. Validation Suite for Conformance of Preprocessing</a></h2>
<p>Items in the test-t, test-c, test-l, and tool directories and this cpp-test.html itself are "Validation Suite for Standard C Conformance of Preprocessing" developed by myself. It tests the level of Standard C (ANSI/ISO/JIS C) conformance for preprocessing in optional compiler systems in detail. It is intended to cover all preprocessing specifications defined in Standard C. It also covers C++ preprocessing. There are many additions addressing issues outside the specification.</p>
<p>As Standard C conformance requirements, not only that compiler systems shall behave correctly, but also that documents shall contain necessary items mentioned accurately. I will explain this in <a href="#3.5">3.5</a>.</p>
<h3>3.1.1. Testcases in test-t Directory</h3>
<p>The test-t directory contains 183 sample text files. Out of 183, 30 are header files, 145 sample text files, and 8 are files gathering small pieces of sample text. All but header files have a name in the *.t format, except some of *.cc. These have nothing to do with compilation phases, but test preprocessing phases. Therefore, they are not necessarily in the correct C programming format. They are rather sample text files for testing preprocessing.</p>
<p>As Standard C implementations can compress preprocessing and compilation to a single process, it is not possible to test preprocessing separately, depending on the implementation. You can say that these *.t samples themselves do not conform Standard C. However, there are many implementations that can be tested by separating preprocessing only. In fact, specifications and problems are clear if they can be separated. The *.t sample files are for those.</p>
<p>*.cc files are samples for C++ preprocessing, provided for some preprocessors which do not accept the files named *.c or *.t as C++ source. Those have the same content with corresponding *.t files.</p>
<p>Among sample text files, there are ones with names starting with n_ (meaning normal), i_ (meaning implementation-dependent), m_ (meaning multi-byte character) and e_ (meaning erroneous.)</p>
<p>Files starting with n_ are samples that do not contain errors, something causing undefined behavior, or implementation-defined parts. Preprocessors conforming to Standard C must be able to process these properly.</p>
<p>Files starting with i_ are samples dependent on implementation-defined specifications regarding character sets assuming the ASCII basic character set. (*) Preprocessors for the implementations with ASCII character set conforming to Standard C must be able to process these properly without errors.</p>
<p>Files starting with e_ are samples that contain some sort of violation of syntax rules or constraints, in other words, errors. Preprocessors conforming to Standard C must be able to diagnose these, but not overlook these.</p>
<p>Files with a number succeeding n_, i_, m_ or e_ are samples that test preprocessing in C90 and common preprocessing specifications in C90 and C99. Among header files, pragmas.h, ifdef15.h, ifdef31.h, ifdef63.h, long4095.h and nest9.h to nest15.h are samples to test C99 preprocessing specifications and others are for common specifications in C90 and C99.</p>
<p>Files with alphabetics other than std or post after n_, i_, e_, or u_, are samples for C99 and C++. n_dslcom.t, n_ucn1.t, e_ucn.t and u_concat.t are samples to test preprocessing specifications common in C99 and C++98, n_bool.t, n_cnvucn.t, n_cplus.t, e_operat.t and u_cplus.t for C++, and the rest for C99.</p>
<p>The file named ?_std.t combines pieces of files in C90 together.<br>
?_std99.t is an equivalent for C99. ?_post.t and ?_post99.t files are bonus files and used for testing <b>mcpp</b> in the 'post-Standard ' mode.</p>
<p>The files named u_*.t are bonus files and the pieces of files to test undefined behaviors. undefs.t combines those as one file. unbal?.h is a header file used in those. unspcs.t tests unspecified behaviors and warns.t does not belong to any of the above, but is the file describing texts for which warnings are desirable. unspcs.t and warns.t are also bonus. Files named m_*.t are samples for several encodings as multi-byte character and wide character sets. It is desirable to process many encodings properly. m_*.t belong to quality test items like u_*.t.</p>
<p>misc.t, recurs.t and trad.t are real bonuses. misc.t is a collection of what is in Standards and other documentation, tests with different results depending on the internal representation of the integer type, tests related to translation phase 5 or 6, tests for enhanced functions, and others. recurs.t is a special case of recursive macro, and trad.t is a sample for the old "Reiser model cpp".</p>
<h3>3.1.2. Testcases in test-c Directory</h3>
<p>There are 133 files in the directory called test-c. 26 of those are header files (24 files are same as the ones in test-t), 102 of them are pieces of sample source files, 3 of them are files which combine pieces of sample source, and the other 2 are files used for automatic testing. Among these, 32 files are bonus sample source files. Source files other than header files are named *.c. This is in the C program format.</p>
<p>Naturally, file names start with n_, i_, m_ or e_. Ones starting with n_ are strictly conforming programs (which does not have any errors nor implementation-dependent portions) in Standard C. Implementations must be able to compile these files correctly without errors and execute them correctly. In case of correct execution, the messages below are displayed.</p>
<pre style="color:navy">
started
success
</pre>
<p>With exceptions of n_std.c and i_std.c, these messages are not displayed. However, only the end message,</p>
<pre style="color:navy">
&lt;End of "n_std.c"&gt;
</pre>
<p>, is displayed. Otherwise, some sort of error message is displayed. Some files starting with i_ are samples of character constant assuming ASCII. Implementations with ASCII character set must be able to compiles these files correctly and execute them correctly as ones starting with n_. Files starting with e_ must be correctly diagnosed by compiler systems at compilation (preprocessing.)</p>
<p>Testing by compilation or execution is the most proper testing method. However, the method detects the existence of error in an implementation, but there are cases it is not clear where errors are. You can give more accurate evaluation by applying *.c files only to a preprocessor and performing testing by looking through the results as far as the implementation allows (*.t files are even more straightforward.)</p>
<p>The files called ?_std.c combines pieces of files.</p>
<p>Files named u_*.c are bonus and pieces of files which test undefined behaviors. undefs.c collects them in one file. unspcs.c tests unspecified behaviors while warns.c does not belong to any of the above, but is the file of texts for which warnings are desirable. unspcs.c and warns.c are a bonus. Those starting with m_ are samples of several multi-byte character encodings.</p>
<p>C99 tests are not included in the test-c directory since there is no compiler proper supporting C99 fully. C++ tests are only in the test-t directory.</p>
<h3>3.1.3. Testcases in test-l Directory</h3>
<p>The test-l directory contains samples for testing translation limits that exceed specifications. All 144 files are bonus. They are a mix of *.c, *.t, and *.h files.</p>
<p>Many *.h files overlap in each directory, test-t, test-c, and test-l. If the duplicate header files are gathered in one directory, the following way of including method is necessary, for example.</p>
<pre style="color:navy">
#include "../test-t/nest1.h"
</pre>
<p>However, the method of searching this type of path list format or files (where to locate the base directory etc.) is all implementation-defined and the compatibility is not guaranteed. In order to avoid this problem, those header files are placed in each directory regardless of duplication (Even the concept of "directory" is excluded from C Standard.)</p>
<h3>3.1.4. Tools in tool Directory</h3>
<p>The tool directory includes tools necessary for automatic testing.</p>
<h3>3.1.5. Testcases for GCC/testsuite in cpp-test Directory</h3>
<p>The files in cpp-test directory are the testcases for GCC/testsuite. They are rewrites of the samples in test-t and test-l directories.</p>
<br>
<h2><a name="3.2" href="#toc.3.2">3.2. Testing Method</a></h2>
<p>When performing tests using the Validation Suite and if a compiler system has options to make closer to Standard C, all should be set (refer to <a href="#6.1">6.1</a> for a concrete example.)</p>
<h3><a name="3.2.1" href="#toc.3.2.1">3.2.1. Manual Testing</a></h3>
<p>Each of test-t and test-c directories contain 2 kinds of samples, big files with multiple pieces put together and small files divided into pieces. If a preprocessor conforms to Standard C well, only big files with multiple pieces put together are necessary to test n_*. However, if the level of conformance is not high, a preprocessor will fall into confusion in the middle with these files and the rest of the items cannot be tested. Therefore, small pieces of files are also provided. Since the number of files gets too large and it is a lot of trouble to do testing if I divide the pieces into too many files, I made a reasonable compromise. Depending on the implementation, even these small pieces of files cannot be processed till the end. In such event, please divide the sample into even smaller pieces to continue testing.</p>
<p>As the #error directive terminates a process depending on implementations, samples testing #error are not included in big files with pieces put together. The #include error also often terminates a process, it is not included in big files with pieces put together.</p>
<h4>3.2.1.1. Viewing the preprocessing result directly</h4>
<p>The *.t samples are used in case a preprocessor is an independent program or that a compiler has an option to output the text after preprocessing. By checking the result of preprocessing these files directly, you can compare if they match correct results written in comments. Since it is possible to view preprocessing results directly, it is possible to make more accurate judgment this way as long as implementations permit.</p>
<p>Many *.c programs include the "defs.h" header. As 2 kinds of assert macro definitions are written in "defs.h", set 0 to 1 in #if 0 for any of these. The first one only includes &lt;assert.h&gt;. The second one is the assert macro which does not abort on an assertion failure. This, of course, is not a correct assert macro, but more convenient in this test.</p>
<p>In multi-byte character processing, behavioral specifications of implementations may differ depending on the runtime environment, thus testing m_* requires attention (refer to <a href="#4.1">4.1</a>.)</p>
<h4>3.2.1.2. Target item to test</h4>
<p>This type of testing has a difficult issue since testing an item may be caught by another failure in an implementation. For example, if &lt;limits.h&gt; has an error and is included to test #if, it is not clear whether &lt;limits.h&gt; or #if is tested. The test which compiles and execute *.c files are more troublesome than one for preprocessing *.t files. If the last result is wrong, it will make it appear that there is some sort of error in the implementation, but not necessarily in the item tested.</p>
<p>I tried to contrive ways to aim the target item in this Validation Suite. However, there is a restriction that the Validation Suite itself has to be portable. In addition, in order to test an item, the correct implementation of other language specification must be assumed. Therefore, the preprocessing item used as this "assumption" is implicitly tested in areas other than the test item that was targeted for the item. Please note such implicit allocation of points also exist in the "allocation of points" which will be described next. It may not be possible to judge whether the sample failed in the test item that it really targeted or by another factor in case an implementation fails a sample process without looking at another test.</p>
<h4>3.2.1.3. Scoring</h4>
<p>Each test item is set by each allocation of points. Marking criteria are also written. Standard C does not have subsets, therefore unless all items match specifications, an implementation cannot be said to be Standard C conforming, strictly speaking. In reality, there are not many such implementations and we cannot help using the measure of Standard C conformance level to evaluate implementations. In addition, as there are large differences in the importance of items, counting the number of passed items will not do, rather a weighting depending on the importance should be applied.</p>
<p>However, this weighting does not have objective criteria, of course. The marking of this Validation Suite was decided by myself and does not have a grounded base. Still, it will be a guideline in evaluating compliance levels for implementations objectively.</p>
<p>n_*, i_*, e_*, and d_* are tests related to Standard conformance and marking for these is in 2 point unit in general. In testing outside of Standards and quality evaluation, marking for q_* is in 2 point and the rest is in 1 point units. Where a diagnostic message should be displayed, no points will be scored in case it is wide of the mark although it is displayed. A partial score may be given to a diagnostic message not absolutely incorrect but rather off the point. An implementation is free to issue diagnostics on correct program if it correctly processes the program, however, wrong diagnostics will be subject to subtraction.</p>
<h3><a name="3.2.2" href="#toc.3.2.2">3.2.2. Automatic Testing by cpp_test</a></h3>
<p>If you compile the cpp_test.c program in the tool directory and run it in the test-c directory, you can test n_*.c and i_*.c for C90 automatically. However, this only scores pass or fail and it does not provide any detail. It does not include tests such as e_*.?. It just takes aim at the conformance level of preprocessors for C90 briefly. No tests regarding C99 are included. That is because most compilers do not support C99 sufficiently yet. *1, *2, *3</p>
<p>How to use cpp_test is, in an example of Visual C++ 2005, as follows.</p>
<pre style="color:navy">
cpp_test VC2005 "cl -Za -TC -Fe%s %s.c" "del %s.exe" &lt; n_i_.lst
</pre>
<p>The second argument and on need to be enclosed by " and " respectively (in case the shell removes ", ", it is necessary to take the measure to enclose the second argument to the last one all together in ' and '.) %s will be replaced by a sample program name without .c such as n_* and i_*.</p>
<p>The first argument: Specifies the name of a compiler system. This must be within 8 bytes and must not include '.'. Files with this name plus .out, .err, or .sum are created.</p>
<p>The second argument: Writes the command to compile.</p>
<p>The third argument and later: Writes the command to delete the files no longer necessary. Multiple of these are allowed.</p>
<p>n_i_.lst is in the test-c directory. It includes the list of n_*.c and i_*.c without .c respectively.</p>
<p>Depending on the implementation, they may start runaway processing some source files. In such an event, change the source name in n_i_.lst to a name which does not exist, none, for example, then run the test again.</p>
<p>By running cpp_test this way, n_*.c and i_*.c are compiled and executed sequentially. The outputs to stderr for sample programs are recorded in the n_*.err and i_*.err files. In addition, the score results are written on a column in VC2005.sum. However, there are only 3 kinds of marking below.</p>
<blockquote>*: Pass<br>
o: Compiles, but the execution result failed.<br>
-: Could not be compiled.</blockquote>
<p>In VC2005.out, the command line that called cpp_test is recorded and so is the message which was output to stdout by the compiler system if any. Messages output to stderr by a compiler system are recorded in VC2005.err if any.</p>
<p>Look at these for more information.</p>
<p>Now, use the following command.</p>
<pre style="color:navy">
paste -d'\0' side_cpp *.sum &gt; cpp_test.sum
</pre>
<p>By doing so, the *.sum files which are test results for each compiler system are combined horizontally to create one table to be recorded in cpp_test.sum. side_cpp is the table side portion where test item titles are written and exists in the test-c directory.</p>
<p>cpp_test.sum that I created this way is located in the doc directory. In <a href="#6.2">6.2</a>, the detail results of manual testing are written. They test more preprocessors than cpp_test.sum. Among those preprocessors, there are some that do not support compiler drivers for any compiler systems. They cannot be tested automatically by cpp_test.</p>
<p>Note:</p>
<p>*1 This cpp_test.c was written based on runtest.c and summtest.c in "Plum-Hall Validation Sampler."</p>
<p>*2 cpp_test.c does not operate with expected behavior if it is compiled on Borland C / bcc32. This is because cpp_test calls system() to redirect stdout and stderr but standard I/O path does not seem to get inherited by the descendant process in bcc32. If cpp_test.c is compiled in Visual C or LCC-Win32, it operates without problems.</p>
<p>*3 m_36_*.c are the tests of encoding which has a byte of 0x5C ('\\') value. cpp-test does not use them, since some systems do not use these encodings.</p>
<h3><a name="3.2.3" href="#toc.3.2.3">3.2.3. Automatic Testing by GCC / testsuite</a></h3>
<h4><a name="3.2.3.1" href="#toc.3.2.3.1">3.2.3.1. TestSuite</a></h4>
<p>GCC source contains something called testsuite. Do 'make check' after compiling the GCC source files, testcases of this testsuite are checked one after another and the results are reported.</p>
<p>My Validation Suite, since V.1.3, was appended the edition which is rewritten so that it could be used as testsuite of GCC. Putting this in testsuite allows automatic checking by 'make check'. While the cpp_test tool in <a href=#3.2.2>3.2.2</a> can test only samples with n_* or i_* as a name, testsuite allows samples which require diagnostic messages such as e_*, w_*, and u_* to be tested automatically. This set of testcases is applicable to cpp0 (cc1, cc1plus) of GCC 2.9x and later and <b>mcpp</b>.</p>
<p>Here, I will explain how to use the Validation Suite in GCC / testsuite.</p>
<p>The cpp-test directory of the Validation Suite is the edition for GCC / testsuite created by rewriting the test-t and test-l directories and cpp-test contains each directory of test-t and test-l.</p>
<p>GCC / testsuite, however, cannot change the execution environment. The files named m_* or u_1_7_* are the testcases for several multi-byte character encodings. Since those testcases need different environments each other for at least GCC 3.3 or former, those are excluded from this testsuite edition. *1</p>
<p>GCC and testsuite specifications have been changed many times thus far and are expected to be changed in the future as well. It may require a partial fix to the Validation Suite accordingly, especially in case of addition or change of diagnostics. However, no extensive fix seems to be necessary so far unless the version of GCC is extremely old. The testcases in cpp-test have been verified in each cpp0 (cc1, cc1plus) of GCC 2.95.3, 3.2, 3.3.2, 3.4.3, 4.0.2 and 4.1.1, and <b>mcpp</b>.</p>
<p>Runtime options cannot be changed in the testsuite depending on the target implementation. As a matter of fact, multiple standards coexist and it is necessary to specify a version of the standard using the 'std=' option. However, this option does not exist in older versions of GCC. Therefore, my testsuite applies to GCC 2.9x and later and <b>mcpp</b> V.2.3 and later.</p>
<p>Testsuite is executed by interpreting the comments in the following format written in the testcase. This is a comment which does not affect tests in other compiler systems.</p>
<pre style="color:navy">
/* { dg-do preprocess } */
/* { dg-error "out of range" "" } */
</pre>
<p>The samples with the comment, dg-error or dg-warning, written test diagnostic messages. Testing multiple compiler systems is supported by writing diagnostic messages of each compiler system with '|' (OR) in-between.</p>
<p>This is executed by the tool called DejaGnu and it is directly a shell-script called runtest. The setup of DejaGnu is written in some files named *.exp. *.exp are the scripts for the tool called Expect. And, Expect is a program written in the command language called Tcl.</p>
<p>Therefore, using testsuite requires these many tools of appropriate versions according to the testsuite. This is same when my Validation Suite is used.</p>
<p>Note:</p>
<p>*1 In fact, GCC does not work properly even if the environment variable is set.</p>
<h4><a name="3.2.3.2" href="#toc.3.2.3.2">3.2.3.2. Installation to TestSuite and Testing</a></h4>
<p>My Validation Suite is used in GCC / testsuite in the following manner.</p>
<p>First, copy the cpp-test directory to an appropriate directory in testsuite of GCC.</p>
<p>The cpp-test directory is the one created by copying necessary files in each directory of test-t and test-l and adding the configuration file cpp-test.exp. The suffix of the files named *.t is mostly changed to .c, the suffix of the files for C++ is changed to .C.</p>
<p>Most samples test the preprocessor only. Since two samples cannot test the preprocessor due to the problems in DejaGnu and Tcl, they are for compiling and running (named *_run.c). These two samples contain the line:</p>
<pre style="color:navy">
{ dg-options "-ansi -no-integrated-cpp" }
</pre>
<p>where -no-integrated-cpp is an option for GCC 3 and 4. GCC 2 does not support the option, which need to be removed in order to test in GCC 2. To accommodate both GCC 2 and GCC 3 or 4, there are two types of files, *_run.c.gcc2 and *_run.c.gcc3, for these two testcases. Link the appropriate one to *_run.c.</p>
<p>Below, I will take an example of GCC 3.4.3 on my Linux. Suppose the source files of GCC 3.4.3 are located in /usr/local/gcc-3.4.3-src. Also, the GCC compilation is done by /usr/local/gcc-3.4.3-objs.</p>
<pre style="color:navy">
cp -r cpp-test /usr/local/gcc-3.4.3-src/gcc/testsuite/gcc.dg
</pre>
<p>This copies files under cpp-test to the gcc.dg directory.</p>
<p>By doing this, if you</p>
<pre style="color:navy">
make bootstrap
</pre>
<p>in /usr/local/gcc-3.4.3-objs to compile the GCC source files and you</p>
<pre style="color:navy">
make -k check
</pre>
<p>then the entire testsuite including testcases in cpp-test will be tested.</p>
<p>Also, testing by using cpp-test only is done as below in the /usr/local/gcc-3.4.3-objs/gcc directory.</p>
<pre style="color:navy">
make check-gcc RUNTESTFLAGS=cpp-test.exp
</pre>
<p>The testsuite logs are recorded in gcc.log and gcc.sum under the ./testsuite directory.</p>
<p>When you do 'make check', depending on the environment, you need to set up the environment variable called DEJAGNULIBS, TCL_LIBRARY as explained in INSTALL/test.html of the GCC source files.</p>
<p>In addition, the environment variable, LANG and LC_ALL, should be C to set the environment to English.</p>
<p>Please note that it is xgcc, cc1, cc1plus, cpp0 etc. generated in the gcc directory that are used in make check at compiling GCC, not gcc, cc1 and such that have already been installed.</p>
<p>Tests can be executed in any directory as follows.</p>
<pre style="color:navy">
runtest --tool gcc --srcdir /usr/local/gcc-3.4.3-src/gcc/testsuite cpp-test.exp
</pre>
<p>Logs are output to the current directory. In this case, what is to be tested is gcc, cc1, and cpp0 which have already been installed. cpp-test requires testsuite as it contains various configuration files for GCC (config.*, *.exp).</p>
<p>The argument 'gcc' of "runtest --tool gcc" should be exactly 'gcc'. If the name of the compiler to be tested is not 'gcc', for example 'cc' or 'gcc-3.4.3', you should make symbolic link so that the compiler is invoked by the name of 'gcc'.</p>
<p>Also, cpp-test contains the testcases for warnings in the cases where it is thought to be desirable for a preprocessor to issue a warning. The GCC preprocessor passes less than a half of those cases, however, not passing does not mean that the behavior is wrong or that the preprocessor was not compiled properly. This is not the issue of being right or wrong, but rather of the "quality" of the preprocessor.</p>
<h4><a name="3.2.3.3" href="#toc.3.2.3.3">3.2.3.3. <b>mcpp</b> Automatic Testing</a></h4>
<p>This cpp-test can test <b>mcpp</b> also. Therefore, substituting the GCC preprocessing with <b>mcpp</b> and calling</p>
<pre style="color:navy">
make check-gcc RUNTESTFLAGS=cpp-test.exp
</pre>
<p>in the gcc directory checks <b>mcpp</b> of Standard mode automatically. Tests can be done also using runtest command in any directory.</p>
<pre style="color:navy">
runtest --tool gcc --srcdir /usr/local/gcc-3.4.3-src/gcc/testsuite cpp-test.exp
</pre>
<p>If <b>mcpp</b> is executed in GCC 3 or 4, all testcases for cpp-test except one should pass. There is another testcase which does not pass when executed in GCC 2. However, it is because gcc calls <b>mcpp</b> with the -D__cplusplus=1 option and not <b>mcpp</b>'s fault.</p>
<p>Please refer to <a href="mcpp-manual.html#3.9.5">mcpp-manual.html#3.9.5</a> and <a href="mcpp-manual.html#3.9.7">mcpp-manual.html#3.9.7</a> for how to substitute the preprocessing with <b>mcpp</b>. To apply the testsuite, <b>mcpp</b> startup needs the -23j options to be set. -2 is an option to enable digraph and -3 is one to enable trigraphs. -j is an option for not adding information such as source lines to diagnostic message output. Do not use other options. Additionally, testsuite can test <b>mcpp</b> in standard mode only, no other.</p>
<p>The method above is done after the make with GCC, however, automatic testing can be done by 'configure' and 'make' of <b>mcpp</b> itself as long as GCC / testsuite is installed and is ready to execute. This case is the easiest as 'make check' automatically performs necessary settings. See the INSTALL in <b>mcpp</b> for this method.</p>
<h4><a name="3.2.3.4" href="#toc.3.2.3.4">3.2.3.4. TestSuite and Validation Suite</a></h4>
<p>GCC has had testsuite for a long time, but very few samples about preprocessing up to V.2.9x. You can see how little attention preprocessing was paid to. The number of testcases for preprocessing increased quite a lot in V.3.x. You can tell preprocessing was given more importance as it was completely changed with up-to-date preprocessor source and documents.</p>
<p>However, these testcases are still quite unbalanced. The causes seem to come from the following nature of testsuite.</p>
<ol>
<li>Collection of bug reports submitted by users. In other words, concentrate on the areas to correct bugs actually detected and to prevent reappearance.<br>
<li>The testcases added for debugging when developers implement new functionalities there.<br>
</ol>
<p>This is the way of debugging special to the open source project and became possible as GCC has been used by many excellent programmers in the world. However, this method might have brought the randomness and imbalance of testcases at the same time.</p>
<p>In addition, most of these testcases are valid only in GCC and cannot be used in other compiler systems. Also, testcases for GCC 3 contain many testcases which cannot be applied to even GCC 2 / cpp. The reason is the differences in preprocessing output spacing and diagnostic messages.</p>
<p>On the other hand, my Validation Suite was originally written by myself only in order to debug my preprocessor and rewritten so that the entire preprocessing specifications are tested. Many samples are organized systematically on the whole.</p>
<p>It will have considerable meaning to add these systematic testcases to GCC / testsuite.</p>
<p>Also, my testsuite edition of Validation Suite is written so that it can test three preprocessors, GCC 2.9x / cpp, GCC 3.x, 4.x / cc1 (cc1plus), and <b>mcpp</b>. In other words, the use of the regular expression facility in DejaGnu and Tcl can absorb the implementation differences in preprocessing output spacing and diagnostic messages. *1</p>
<p>Below are the results of applying the testsuite edition of the Validation Suite to these three preprocessors (tested <b>mcpp</b> in March, 2007, others in October, 2006).</p>
<p>Below is the case where the preprocessor is replaced by <b>mcpp</b> V.2.6.3 in GCC 4.1.1.</p>
<pre style="color:navy">
=== gcc Summary ===
# of expected passes 264
# of unexpected failures 1
# of expected failures 4
/usr/local/bin/gcc version 4.1.1
</pre>
<p>There is one failure due to a lack of the universal-character-name &lt;=&gt; multi-byte character conversion implementation in C++98.</p>
<p>Here is the GCC 3.2 / cc1 case.</p>
<pre style="color:navy">
=== gcc Summary ===
# of expected passes 216
# of unexpected failures 51
# of unexpected successes 2
# of expected failures 2
/usr/local/bin/gcc version 3.2
</pre>
<p>Most of failures are due to a missing warning.</p>
<p>GCC 4.1.1 / cc1 is almost the same.</p>
<pre style="color:navy">
=== gcc Summary ===
# of expected passes 214
# of unexpected failures 53
# of unexpected successes 2
# of expected failures 2
/usr/local/bin/gcc version 4.1.1
</pre>
<p>Here is the GCC 2.95.3 / cpp0 case.</p>
<pre style="color:navy">
=== gcc Summary ===
# of expected passes 181
# of unexpected failures 87
# of unexpected successes 3
# of expected failures 1
gcc version 2.95.3 20010315 (release)
</pre>
<p>There are less warnings than GCC 3, 4 / cc1. There are also some diagnostic messages that are off the point. Half of new C99 and C++98 specifications have not been implemented yet, either.</p>
<p>The number of items differ among different versions of GCC, since multiple failures can occur in one testcase.</p>
<p>Note:</p>
<p>*1 This makes dg script in my testcases difficult to read with frequent use of \ and symbols. The regular expression processing in DejaGnu and Tcl has a considerable number of peculiarities and flaws requiring ingenuity to perform all the automatic testing achieved on multiple compiler systems. Currently, however, runtime options used in testcases for those compiler systems have to be common.</p>
<br>
<h2><a name="3.3" href="#toc.3.3">3.3. Violation of syntax rules or Constraints and Diagnostic Messages</a></h2>
<p>Standard C implementations must certainly process correct source correctly, but they also must issue diagnostic messages for erroneous source. Standard C also contains portions where behavior specifications are up to the implementation or not defined. They are as below in summary. *1</p>
<ol>
<li>Correct programs and data whose outcomes are the same in every implementation.<br>
<br>
<li>Correct programs and data whose process methods are not specified. These do not need to be described in documents and the results are called unspecified behavior.<br>
<br>
<li>Correct programs and data whose processing is up to implementations. These specifications must be mentioned in documents by each implementation. These results are called implementation-defined behavior.<br>
<br>
<li>Programs or data that are erroneous or not portable and their processing is not defined as specifications at all. Implementations may or may not output diagnostic messages. They may process them as some sort of valid programs. These results are called undefined behavior.<br>
<br>
<li>Erroneous programs or data for which implementations must issue diagnostic messages. There are violations of syntax rules and violations of constraint among these. *2<br>
</ol>
<p>Among these, programs and data in 1 only are called strictly conforming (it is interpreted that 2 and 3 may be included if their results do not differ depending on implementation or special cases.)</p>
<p>Programs and data in 1, 2, and 3 only are called conforming programs.</p>
<p>How to issue diagnostic messages is implementation-defined. Supposedly, one or multiple diagnostic messages of some sort are issued for one translation unit that includes some sort of violations of syntax rules or constraints. It is up to the implementation whether diagnostic messages should be issued for the programs with no violation of syntax rules or constraint. However, strictly conforming programs or conforming programs matching implementation-defined or unspecified specifications for the implementation must be able to be processed correctly until the end.</p>
<p>Violations of syntax rules or constraints are called an "error" in this document. Among e_* files in this Validation Suite, there are many which include multiple errors. In scoring below, it is expected that a compiler system issues one or more diagnostic messages. However, there may be compiler systems that issue just one diagnostic message (such as "violation of syntax rules or constraints") for one translation unit no matter how many errors there are. In addition, there may be compiler systems that get confused after an error. These types of problems are of "quality", but not of Standard conformance level. Please make samples into pieces and test them again as needed. The problems of quality will be discussed in <a href=#4>4</a> separately.</p>
<p>Note:</p>
<pre>
*1 C90 3 Definitions of Terms
C90 4 Compliance
C90 5.1.1.3 Diagnostics
C99 3 Terms, definitions, and symbols
C99 4 Conformance
C99 5.1.1.3 Diagnostics
</pre>
<p>*2 Although C++98 differs from C90 or C99 in these terms, it does not much differ in the meanings.</p>
<br>
<h2><a name="3.4" href="#toc.3.4">3.4. Details</a></h2>
<p>Each test item is explained one by one below. This is also a description of Standard C preprocessing itself. The specifications in common with K&amp;R 1st are not explained again. Item numbers are common in *.t and *.c files.</p>
<h3><a name="3.4.1" href="#toc.3.4.1">3.4.1. Trigraphs</a></h3>
<h4>n.1.1. 9 trigraph sequences</h4>
<p>As there are 9 characters not included in the Invariant Code Set of ISO 646:1983 among the basic character set in C, these can be written in source using 3 character sequence below. This is a new specification introduced in C90. *1</p>
<blockquote>
<table>
<tr><th>??=</th><td>#</td></tr>
<tr><th>??(</th><td>[</td></tr>
<tr><th>??/</th><td>\</td></tr>
<tr><th>??)</th><td>]</td></tr>
<tr><th>??'</th><td>^</td></tr>
<tr><th>??&lt;</th><td>{</td></tr>
<tr><th>??!</th><td>|</td></tr>
<tr><th>??&gt;</th><td>}</td></tr>
<tr><th>??-</th><td>~</td></tr>
</table>
</blockquote>
<p>Equivalent characters replace these 9 trigraph sequences in translation phase 1. On systems where you can type in these 9 characters on keyboard, it is not necessary to use trigraphs, of course. However, it is necessary for preprocessing conforming Standard C to be able to do trigraph conversion on even those systems.</p>
<p>Scoring: 6. 6 points if all 9 are processed correctly. Each trigraph which cannot be processed properly, 2 points are reduced with 0 at the lowest limit.</p>
<p>Note:</p>
<pre>
*1 C90 5.2.1.1 Trigraph sequences
C90 5.1.1.2 Translation phases
C99 5.2.1.1 Trigraph sequences
C99 5.1.1.2 Translation phases
</pre>
<h4>n.1.2. Trigraph sequences in control lines</h4>
<p>Since trigraph conversion is performed prior to tokenization in translation phase 3 or control line processing in phase 4, trigraphs can be written wherever on a control line.</p>
<p>Scoring: 2.</p>
<h4>n.1.3. Only 9 trigraphs</h4>
<p>There are only 9 trigraphs mentioned above, therefore sequences starting with ?? other than those are never translated into another character nor ?? can be skipped. Preprocessing must be able to handle the case of sequences with a trigraph and ?'s which are not trigraphs.</p>
<p>Scoring: 2.</p>
<h3><a name="3.4.2" href="#toc.3.4.2">3.4.2. Line Splicing by &lt;backslash&gt;&lt;newline&gt;</a></h3>
<p>In case there is a \ at the end of a line and a &lt;newline&gt; immediately afterward, this sequence of &lt;backslash&gt;&lt;newline&gt; is deleted in translation phase 2 unconditionally. As a result, 2 lines are connected. In Standards, the line on a source file is called a physical line to distinguish it while the line connected by removing &lt;backslash&gt;&lt;newline&gt; (if any) is called a logical line. Processing in translation phase 3 is performed with this logical line as subject. *1</p>
<p>In K&amp;R 1st, the #define line and string constants can continue on the next source line using &lt;backslash&gt;&lt;newline&gt;, but other cases are not mentioned. Actual implementations allow other control lines may connected, not only #define.</p>
<p>Note:</p>
<pre>
*1 C90 5.1.1.2 Translation phases
C99 5.1.1.2 Translation phases
</pre>
<h4>n.2.1. Between a parameter list and replacement list on the #define line</h4>
<p>The #define line connections are accepted in K&amp;R 1st and most of implementations.</p>
<p>Scoring: 4. 4 points for processing correctly and 0 point otherwise.</p>
<h4>n.2.2. Inside a parameter list on the #define line</h4>
<p>There are some implementations which cannot handle &lt;backslash&gt;&lt;newline&gt; in unusual places such as inside a parameter list even on the #define line.</p>
<p>Scoring: 2.</p>
<h4>n.2.3. Inside a string literal</h4>
<p>&lt;backslash&gt;&lt;newline&gt; inside a string literal has been supported since K&amp;R 1st.</p>
<p>Scoring: 2.</p>
<h4>n.2.4. Inside an identifier</h4>
<p>In Standard C, &lt;backslash&gt;&lt;newline&gt; must be removed unconditionally even if it is inside an identifier or anywhere.</p>
<p>Scoring: 2.</p>
<h4>n.2.5. &lt;backslash&gt; as a trigraph</h4>
<p>&lt;backslash&gt; is not only \, but also ??/ as a trigraph. ??/ in source is converted into \ in translation phase 1, it is obviously \ itself in phase 2.</p>
<p>Scoring: 2.</p>
<h3><a name="3.4.3" href="#toc.3.4.3">3.4.3. Comments</a></h3>
<p>In translation phase 3, a logical line is broken into pp-tokens and white spaces. A comment is converted into a single space at that time. *1</p>
<p>Here, implementations may convert consecutive white spaces (including comments) into a single space. However, &lt;newline&gt; is not converted and stays as is in any case. That's because the process of preprocessing directive in the next phase 4 is subject to this "line."</p>
<p>In case the comment expands over lines, line splicing is performed virtually by a comment.</p>
<p>Note:</p>
<pre>
*1 C90 5.1.1.2 Translation phases
C99 5.1.1.2 Translation phases
</pre>
<h4>n.3.1. Conversion into one space</h4>
<p>In the old cpp so-called Reiser type, comments functioned as token separators only internally in cpp and were removed before output. By taking advantage of it, there was a method of using comments for token concatenation. However, this specification derailed K&amp;R 1st and was clearly rejected by Standard C. In Standard C, the ## operator is used for token concatenation.</p>
<p>Scoring: 6.</p>
<h4>n.dslcom. Comment by //</h4>
<p>From K&amp;R 1st to C90, comments started with /* and ended with */. *1</p>
<p>However, C99 started supporting C++ style of comment, //. *2</p>
<p>Scoring: 4.</p>
<p>In C90, this should be processed as just a sequence of pp-token '/' and '/', not a comment. However, as implementations which handle // as comments even prior to C99 were common, <b>mcpp</b> treats this as a comment and issues a warning in C90 mode.</p>
<p>Note:</p>
<pre>
*1 C90 6.1.9 Comments
*2 C99 6.4.9 Comments
</pre>
<h4>n.3.3. Comment processing prior to pp-directive processing</h4>
<p>The preprocessing directive starting with # is for a "line", but this "line" is not necessarily a physical line in source. It could be a logical line combined by &lt;backslash&gt;&lt;newline&gt; or the "line" which extend over multiple physical and logical lines by a comment. This is not surprising if you think about the order of translation phase 1 through 4.</p>
<p>Scoring: 4.</p>
<h4>n.3.4. Comments and &lt;backslash&gt;&lt;newline&gt;</h4>
<p>There are pp-directive lines that extend over some physical lines by both &lt;backslash&gt;&lt;newline&gt; and a comment. Preprocessors that do not implement translation phases properly cannot handle this correctly.</p>
<p>Scoring: 2.</p>
<h3><a name="3.4.4" href="#toc.3.4.4">3.4.4. Special Tokens (digraphs) and Characters (UCN)</a></h3>
<p>In C90/Amendment 1 (1994), alternative spelling called digraph was added for some of operators and punctuators. *1</p>
<p>In C99, a character code called UCN (universal character sequence) was added. *2</p>
<p>In e.4.?, token errors are covered.</p>
<p>Note:</p>
<pre>
*1 Amendment 1/3.1 Operators, 3.2 Punctuators (added to ISO 9899/6.1.5, 6.1.6)
C99 6.4.6 Punctuators
*2 C99 6.4.3 Universal character names
</pre>
<h4>n.4.1. Digraph spelling in a preprocessing directive line</h4>
<p>Digraphs are handled as tokens (pp-tokens.) '%:' is another spelling for '#'. It can certainly be used as the first pp-token of a preprocessing directive line or as a string operator.</p>
<p>Scoring: 6.</p>
<h4>n.4.2. Digraph spelling stringizing</h4>
<p>Different from trigraphs, as digraphs are tokens (pp-tokens), they are stringized as they are in spelling without being converted (meaningless specification.)</p>
<p>Scoring: 2.</p>
<h4>n.ucn1. UCN recognition 1</h4>
<p>UCN is recognized in string literals, character constants, and identifiers.</p>
<p>Scoring: 8. 4 points if UCN in a string literal passes preprocessing as is. 2 points each if UCN is processed correctly in a character constant or identifier. No good if UCN is not recognized and output as is.</p>
<h4>n.ucn2. UCN recognition 2</h4>
<p>UCN can be used inside a pp-number as well. However, it has to disappear from a number-token by the end of preprocessing. This specification exists in C99, not in C++.</p>
<p>Scoring: 2.</p>
<h4>e.ucn. UCN errors</h4>
<p>UCN must be 8 digit hexadecimal if it starts with \U or 4 digit hexadecimal if it starts with \u.</p>
<p>UCN must not be in the range between [0..9F] and [D800..DFFF]. However, 24($), 40(@), and 60(`) are valid.</p>
<p>Scoring: 4. 1 point each for each correct diagnosis regarding 4 samples.</p>
<h4>e.4.3. Empty character constants</h4>
<p>Even sequences not in C token format are also recognized as pp-tokens. Therefore, there are not many error cases in tokenization of preprocessing.</p>
<p>However, there are some cases other than this which become undefined behavior (refer to <a href="#4.2">4.2</a>.)</p>
<p>Empty character constants are violations of syntax rules in a preprocessing #if line or compiling. *1</p>
<p>Scoring: 2.</p>
<p>Note:</p>
<pre>
*1 C90 6.1.3.4 Character constants -- Syntax
C99 6.4.4.4 Character constants -- Syntax
</pre>
<h3><a name="3.4.5" href="#toc.3.4.5">3.4.5. Spaces and Tabs on a Preprocessing Directive Line</a></h3>
<p>Spaces, tabs, vertical-tabs, form-feeds, carriage-returns, and new-lines are all white spaces. White spaces that are not in string literals, character constants, header names, or comments usually have a meaning as a token separator. However, new lines that remain until translation phase 4 are special and become a pp-directive separator. There are slight restrictions in white spaces that can be used in pp-directive lines.</p>
<h4>n.5.1. Spaces and tabs before and after #</h4>
<p>In Standard C, spaces and tabs before and after the #, which is the first pp-token on a preprocessing directive line are guaranteed to end in the same results whether they exist or not (*1.) In K&amp;R 1st, this is not clear and there were actually implementations which do not accept spaces and tabs before and after #.</p>
<p>In K&amp;R 1st, it is interpreted that spaces and tabs after that in the line are accepted as just a token separator as in Standard C.</p>
<p>However, in the case where there are white spaces other than spaces and tabs on the pp-directive line, it is undefined (refer to <a href="#u.1.6">u.1.6</a> of 4.2.)</p>
<p>Scoring: 6.</p>
<p>Note:</p>
<pre>
*1 C90 6.8 Preprocessing directives -- Description, Constraints
C99 6.10 Preprocessing directives -- Description, Constraints
</pre>
<h3><a name="3.4.6" href="#toc.3.4.6">3.4.6. #include</a></h3>
<p>#include is the most basic pp-directive since K&amp;R 1st. However, the specifications on this directive in Standard C have more undefined and implementation-defined portions (*1.) The reasons are below.</p>
<ol>
<li>A pp-token called header-name which is an argument for this directive is dependent on OS file systems and difficult to standardize.<br>
<li>"Standard" location to search files depends on implementations.<br>
<li>Even the header-name in the format similar to the string literal enclosed by ", " is another pp-token different from a string literal as \ is not an escape character. Furthermore, a header-name enclosed by &lt; and &gt; is the most irregular pp-token.<br>
</ol>
<p>In n.6.*, the most basic test below is not performed by different categories. This is included with other test items on the premise that not being able to process this is out of the question since many tests will not be able to be performed.</p>
<pre style="color:navy">
#include &lt;ctype.h&gt;
#include "header.h"
</pre>
<p>Note:</p>
<pre>
*1 C90 6.8.2 Source file inclusion
C99 6.10.2 Source file inclusion
</pre>
<h4>n.6.1. Standard header include in 2 formats</h4>
<p>There are 2 formats in header-names. The difference is only that the format enclosed by &lt; and &gt; searches the header from a implementation-defined specific location (may be multiple locations) while the one enclosed by " and " searches the source file first in implementation-defined method and performs the same process as the one enclosed by &lt; and &gt; upon failure. Therefore, the format enclosed by " and " can include standard headers as well. This point was the same in K&amp;R 1st.</p>
<p>In Standard C, same standard headers can be included many times. Either way, however, this is not a preprocessor issue, but on how to write standard headers (refer to <a href="#5.1.1">5.1.1</a>.)</p>
<p>Scoring: 10. 4 scores if only one of 2 samples is processed.</p>
<h4>n.6.2. Header-name by a macro - Part 1</h4>
<p>In K&amp;R 1st, no macro could be used on the #include line, however, it was officially permitted in Standard C. In case the #include argument does not match either of the 2 formats, the macro included there is expanded. The result must match either one of the 2 formats.</p>
<p>This specification has something subtle. For example, how should the following source be handled?</p>
<pre style="color:navy">
#define MACRO header.h&gt;
#include &lt;MACRO
</pre>
<p>Should MACRO be expanded first as below?</p>
<pre style="color:navy">
#include &lt;header.h&gt;
</pre>
<p>Or, should this be an error as &gt; matching &lt; does not exist prior to macro expansion?</p>
<p>I cannot think that Standards are written with this level of detail in mind. Therefore, I believe it is more straightforward to handle &lt; and &gt; as quotation delimiters similar to " and ". Though, expanding macros first cannot be said to be against Standards. This Validation Suite does not include tests which dig holes of this type of specifications.</p>
<p>Scoring: 6.</p>
<h4>n.6.3. Header-name by a macro - Part 2</h4>
<p>This is not so interesting, but it does not have to be a single macro.</p>
<p>Scoring: 2.</p>
<h3><a name="3.4.7" href="#toc.3.4.7">3.4.7. #line</a></h3>
<p>The #line pp-directive is not usually used by a user, but used to pass along the filename of the original source and line numbers occasionally in case another tool or something pre-preprocesses source (not necessarily C.) As this has been around since K&amp;R 1st, it must have some purpose in its own way traditionally.</p>
<p>In addition, #line or its variant is used to pass along a filename and line number information to a compiler proper for preprocessor output in general. However, this is not defined as a specification.</p>
<p>The filename and line number specified in #line become the value of predefined macro, <tt>__FILE__</tt> and <tt>__LINE__</tt> (in addition, <tt>__LINE__</tt> will be incremented for every physical line.) *1</p>
<p>Note:</p>
<pre>
*1 C90 6.8.4 Line control
C99 6.10.4 Line control
C90 6.8.8 Predefined macro names
C99 6.10.8 Predefined macro names
</pre>
<h4>n.7.1. Line number and filename specification</h4>
<p>#line specifying a line number and a filename was in K&amp;R 1st, but the filename was not surrounded by " and ".</p>
<p>A filename is a string literal in Standard C, but a different token from the header-name for #include. Strictly speaking, it has subtle problems in \ handling and the like. However, no problems will arise for valid source (it is fortunate that the filename for #line are not in the &lt;stdio.h&gt; format.)</p>
<p>Scoring: 6.</p>
<h4>n.7.2. No filename may be specified</h4>
<p>The filename argument is optional and it does not have to exist. This is same as K&amp;R 1st (undefined if no line number is specified.)</p>
<p>Scoring: 4.</p>
<h4>n.7.3. Line number and filename specification by a macro</h4>
<p>In K&amp;R 1st, the #line argument could not use a macro. This is permitted in Standard C.</p>
<p>Scoring: 4.</p>
<h4>n.line. Line number range</h4>
<p>The line number range for the #line directive was [1.32767] in C90, but was extended to [1.2147483647] in C99.</p>
<p>Scoring: 2.</p>
<h4>e.7.4. Filename in wide string literal</h4>
<p>The filename argument must be a string literal. This is not so interesting, but wide string literals become a violation of constraint (pp-tokens other than that are undefined for some reasons. This is an imbalanced specification.)</p>
<p>Scoring: 2.</p>
<h3><a name="3.4.8" href="#toc.3.4.8">3.4.8. #error</a></h3>
<p>#error is a directive newly introduced in Standard C. It displays an error message that includes an argument as a part at preprocessing. In old implementations, there were some with directives such as #assert. However, these were not standard.</p>
<p>It is not specified that #error should end a process. According to the Rationale, it was not specified since the Standard cannot make requirements to that extent. However, it says that ending a process is the ANSI C committee's intention. *1</p>
<p>Note:</p>
<pre>
*1 C90 6.8.5 Error directive
ANSI C Rationale 3.8.5
C99 6.10.5 Error directive
C99 Rationale 6.10.5
</pre>
<h4>n.8.1. No macro expansion</h4>
<p>Macros on the #error line are not expanded. Macros which are expanded on control lines are only for #if (#elif), #include, and #line. *1</p>
<p>Scoring: 8. 2 points for processing by expanding macros. Whether to terminate the process does not matter.</p>
<p>Note:</p>
<pre>
*1 C90 6.8 Preprocessing directives -- Semantics
C99 6.10 Preprocessing directives -- Semantics
</pre>
<h4>n.8.2. Optional message</h4>
<p>This is not so interesting, but the #error line argument is optional and it does not have to exist.</p>
<p>Scoring: 2.</p>
<h3><a name="3.4.9" href="#toc.3.4.9">3.4.9. #pragma, _Pragma() operator</a></h3>
<p>#pragma was also introduced in Standard C. Extended directives which are unique to an implementation are all supposed to be implemented by #pragma sub-directives. *1</p>
<p>Note:</p>
<pre>
*1 C90 6.8.6 Pragma directive
C99 6.10.6 Pragma directive
</pre>
<h4>n.9.1. No error for unrecognized #pragma</h4>
<p>Different #pragma sub-directives are recognized for each implementation. As it is not possible to write portable programs if each unrecognized #pragma becomes an error, #pragma which cannot be recognized by implementations is ignored. In preprocessing, only the #pragma recognizably regarding preprocessing is processed while the rest of #pragma is all passed to compiler proper as is.</p>
<p>Scoring: 10. #pragma must not be an error, however, it is acceptable to issue a warning. 10 points for the case that preprocessing does not issue an error but the compiler proper does, which should not occur, though it is not a mistake in preprocessing even if it does. 0 points as a matter of convenience in case this distinction does not exist in the implementation where preprocessors are not independent.</p>
<h4>n.pragma. _Pragma() operator</h4>
<p>C99 introduced the _Pragma() operator which has the same effect as #pragma but can be written in a macro definition as opposed to #pragma.</p>
<p>In addition, when a pp-token following a pragma has a standard feature if it is STDC and macro expansion is prohibited in this case. In other cases, however, whether macro expansion is applicable is implementation-defined.</p>
<p>Scoring: 6.</p>
<h4>e.pragma. _Pragma() argument is string literal</h4>
<p>The _Pragma() operator argument must be a string literal.</p>
<p>Scoring: 2.</p>
<h3><a name="3.4.10" href="#toc.3.4.10">3.4.10. #if, #elif, #else, and #endif</a></h3>
<p>#if, #else, and #endif have been supported since K&amp;R 1st. (*1) In the implementations which cannot use #if, none of n.10.1, n.11.*, n.12.*, n.13.*, e.12.*, and e.14.* can be processed as well as many other tests.</p>
<p>Note:</p>
<pre>
*1 C90 6.8.1 Conditional inclusion
C99 6.10.1 Conditional inclusion
</pre>
<h4>n.10.1. #elif support</h4>
<p>#elif was added in Standard C. By using this, we can avoid illegibility of multiple #if nesting.</p>
<p>Macros can be used in the #if expression. The identifier not defined as a macro is evaluated as 0.</p>
<p>In Standards, #if to the corresponding #endif is called a #if section and a block divided by #if (#ifdef, #ifndef), #elif, #else, or #endif in the section is called a #if group. The #if and #elif line expressions are called a #if expression.</p>
<p>Scoring: 10.</p>
<h4>n.10.2. pp-token processing in the #if group which is skipped</h4>
<p>In the #if group skipped, preprocessing directives are not processed other than checking #if, #ifdef, #ifndef, #elif, #else, and #endif in order to trace the correspondence of #if group and macros are not expanded, either.</p>
<p>However, tokenization takes place. First of all, that is because C source is a sequence of comments and pp-tokens from the beginning till the end. Secondly, it is necessary to process comments at least in order to check the corresponding relation of #if and others, and in order to check if /* and others are comment symbols, they must be made sure that they are not in a string literal or character constant.</p>
<p>Scoring: 6.</p>
<h3><a name="3.4.11" href="#toc.3.4.11">3.4.11. #if defined</a></h3>
<p>An operator, defined, was introduced for #if expressions in Standard C. This integrates #ifdef and #ifndef into #if (#elif) and prevents illegibility of multiple #ifdef nesting.</p>
<h4>n.11.1. #if defined</h4>
<p>The operand for the defined operator is an identifier. Both enclosing it by ( and ) and not doing so are supported as a writing style. Both mean the same and are evaluated as 1 if an operand is defined as a macro, 0 otherwise.</p>
<p>Scoring: 8. 2 scores if only one of 2 #if sections is processed.</p>
<h4>n.11.2. defined is an unary operator</h4>
<p>defined is one of the operators and provides either value of 1 or 0. Therefore, it is possible to do an operation on its result and another expression.</p>
<p>Scoring: 2.</p>
<h3><a name="3.4.12" href="#toc.3.4.12">3.4.12. #if Expression Type</a></h3>
<p>The #if expressions were vaguely defined as constant expressions in K&amp;R 1st and their types were not clear. In C90, the #if expressions are integer constant expressions and it was made clear that int and unsigned int have the same internal representation as long and unsigned long respectively. In other words, the #if expressions including sub-expressions in them are all evaluated in long and unsigned long. To restate, they are handled as if constant tokens all had L or UL suffix. *1</p>
<p>In addition, the #if expression type became the maximum integer type for each implementation in C99. typedef to the type names, intmax_t and uintmax_t, is applied to this type in the standard header called &lt;stdint.h&gt;. As long long is required in C99, the #if expression type is long long or a wider size. Suffixes, LL (ll)/ULL (ull), are used to write long long/unsigned long long constants. *2</p>
<p>A length modifier, %ll, and an appropriate conversion specifier (such as %lld, %llu, and %llx) are used to display the value of long long/unsigned long long in printf(). The length modifier to display the value of intmax_t or uintmax_t is %j (%jd, %ju, %jx and others.) *3</p>
<p>Note:</p>
<pre>
*1 C90 6.8.1 Conditional inclusion -- Semantics
*2 C99 6.10.1 Conditional inclusion -- Semantics
*3 C99 7.19. 6.1 The fprintf function -- Description
</pre>
<h4>n.12.1. long type #if expressions</h4>
<p>In C90, constant expressions of the long type in the #if expressions must be evaluated.</p>
<p>Scoring: 6.</p>
<h4>n.12.2. unsigned long type #if expressions</h4>
<p>In C90, constant expressions of the unsigned long type in the #if expressions must be evaluated.</p>
<p>Scoring: 4.</p>
<h4>n.12.3. Octal numbers</h4>
<p>Octal numbers must be evaluated in the #if expressions. It is similar in K&amp;R 1st, however the constant exceeding the maximum value of long was evaluated as a negative value of long in K&amp;R 1st while it is evaluated as unsigned long in C90. *1</p>
<p>Scoring: 4. 2 points when recognized as an octal number but evaluated as a negative number or overflow. 0 points for not recognized as an octal number.</p>
<p>Note:</p>
<pre>
*1 C90 6.1.3.2 Integer constants
C99 6.4.4.1 Integer constants
</pre>
<h4>n.12.4. Hexadecimal numbers</h4>
<p>Hexadecimal numbers must be also evaluated in the #if expressions. It is similar in K&amp;R 1st, however the difference is that the constant exceeding the maximum value of long was evaluated as a negative value of long in K&amp;R 1st while it is evaluated as unsigned long in C90.</p>
<p>Scoring: 4. 2 points when recognized as a hexadecimal number but evaluated as a negative number or overflow. 0 points for not recognized as a hexadecimal number.</p>
<h4>n.12.5. Suffixes L and l</h4>
<p>Constant tokens with a suffix, L or l, must be also evaluated in the #if expressions. The constants not exceeding the maximum value of long are the same in K&amp;R 1st also. This suffix does not matter in preprocessing for evaluation.</p>
<p>Scoring: 2.</p>
<h4>n.12.6. Suffixes U and u</h4>
<p>Constant tokens with a suffix, U or u, must be also evaluated in the #if expressions. This was the notation not supported in K&amp;R 1st and was officially accepted in C90.</p>
<p>Scoring: 6.</p>
<h4>n.12.7. Negative numbers</h4>
<p>Negative numbers must be also handled in the #if expressions. This is a specification since K&amp;R 1st.</p>
<p>Scoring: 4.</p>
<h4>e.12.8. Constant token value out of range to represent</h4>
<p>In C90, it is an violation of constraint or error, to put it simply, if the integer constant token value which appears in the #if expression is not in the range which can be represented in long or unsigned long. This is not something directly defined as a specification for the #if expressions, however, there is a specification regarding constant expressions in general. The #if expressions are not exceptions, either. *1</p>
<p>The character constant overflow is tested in e.32.5, e.33.2, and e.35.2.</p>
<p>Scoring: 2.</p>
<p>Note:</p>
<pre>
*1 C90 6.4 Constant expressions -- Constraints
C99 6.6 Constant expressions -- Constraints
</pre>
<h4>n.llong. long long/unsigned long long evaluation</h4>
<p>At least, constant expressions in long long/unsigned long long must be evaluated in the #if expressions in C99.</p>
<p>The constant exceeding the maximum value of long long is evaluated as unsigned long long.</p>
<p>Suffixes, LL, ll, ULL, and ull, are added in C99. *1</p>
<p>Scoring: 10. 2 points each for processing each of 5 samples correctly.</p>
<p>Note:</p>
<pre>
*1 C99 6.4.4.1 Integer constants
C99 6.6 Constant expressions -- Constraints
</pre>
<h4>e.intmax. Operation result out of intmax_t range</h4>
<p>The constant or constant expression which exceeds the intmax_t or uintmax_t range is a violation of constraint in the #if expression for C99.</p>
<p>Scoring: 2. In case there is no &lt;stdint.h&gt;, it is acceptable to write a macro appropriately.</p>
<h3><a name="3.4.13" href="#toc.3.4.13">3.4.13. #if Expression Evaluation</a></h3>
<p>The #if expressions are a kind of integer constant expressions. Compared with usual integer constant expressions, there are differences below.</p>
<ol>
<li>In C90, int and unsigned int are evaluated as if they have the same internal representation as long and unsigned long respectively. In C99, int and long are evaluated as if they have the same internal representation as intmax_t. unsigned int and unsigned long are evaluated as if they have the same internal representation as uintmax_t.<br>
<br>
<li>The 'defined' operator is supported.<br>
<br>
<li>All identifiers remained after macro expansion are evaluated as 0. *1<br>
<br>
<li>As there exists no keywords in preprocessing, identifiers with same name as a keyword are treated as just an identifier. Therefore, neither cast nor sizeof can be used.<br>
</ol>
<p>Function calls and comma operators cannot be used in integer constant expressions. Since constant expressions are not variables, no assignments, increments, decrements, nor arrays can be used.</p>
<p>In n.13, evaluation rules common with generic integer constant expressions are tested. Among these, n.13.5 is different from K&amp;R 1st. n.13.6 is the area which was all different in pre-Standard #if expressions. n.13.13 and n.13.14 were not clear in K&amp;R 1st. The rest is unchanged since K&amp;R 1st. *2</p>
<p>The n.13.* uses only small values so that this rule can be tested in the implementations where only int values can be evaluated in the #if expressions. The defined operator and &gt;= are not in n.13, but somewhere else.</p>
<p>Note:</p>
<p>*1 In C++ Standard, 'true' and 'false' are treated differently and evaluated as 1 and 0 respectively. These are not macros, but keywords; however, they are treated as boolean literals in preprocessing.</p>
<pre>
*2 C90 6.3 Expressions
C90 6.4 Constant expressions
C99 6.5 Expressions
C99 6.6 Constant expressions
</pre>
<h4>n.13.1. &lt;&lt; and &gt;&gt;</h4>
<p>Bit shift operations have no troublesome issues regarding positive numbers, at least.</p>
<p>Scoring: 2.</p>
<h4>n.13.2. ^, |, and &amp;</h4>
<p>Since the bit pattern of the same value is same despite of CPUs or implementations in positive integers as long as it is in the type range, operations such as ^, |, and &amp; which may appear to be dependent on CPU specifications have the exact same results in that range on any implementation.</p>
<p>Scoring: 2.</p>
<h4>n.13.3. || and &amp;&amp;</h4>
<p>All implementations should be able to process these.</p>
<p>Scoring: 4.</p>
<h4>n.13.4. ? :</h4>
<p>All implementations should be able to process these, too (it seemed so, but not so in reality.)</p>
<p>Scoring: 2.</p>
<h4>n.13.5. No usual arithmetic conversion in &lt;&lt; and &gt;&gt;</h4>
<p>Usual arithmetic conversions are performed in many binary operators in order to match types on both sides. In K&amp;R 1st, usual arithmetic conversions were performed in shift operators while this is not the case in Standard C. This is an adequate specification considering that the right side value is always small positive number and that a bit pattern changes also if a negative number is converted into a positive number in internal representation other than 2's complement which creates a confusion. *1</p>
<p>Scoring: 2.</p>
<p>Note:</p>
<pre>
*1 C90 6.3.7 Bitwise shift operators -- Semantics
C99 6.5.7 Bitwise shift operators -- Semantics
</pre>
<p>It says point by point, "If both operands have arithmetic type, the usual arithmetic conversions are performed on them.", regarding many binary operators. However, this is not mentioned as far as &lt;&lt; and &gt;&gt; are concerned. There is an explanation regarding this in C89 Rationale 3.3.7 (C99 Rationale 6.5.7.)</p>
<h4>n.13.6. Conversion from a negative number into a positive number by usual arithmetic conversion</h4>
<p>Usual arithmetic conversions are applied to operands on both sides for binary operators such as *, /, %, +, -, &lt;, &gt;, &lt;=, &gt;=, ==, !=, &amp;, ^, and |, in order to match types on both sides. Same for the second and third operands for tertiary operator, ? :. Therefore, if one side is an unsigned type, the other is converted into an unsigned type that causes a negative value to be converted into a positive number.</p>
<p>In standard integer constant expressions, the integer promotion is applied to the operand before the usual arithmetic conversion if each operand is in the integer type shorter than int. However, in the #if expressions, no integer promotion occurs since all operands are handled as the same size type.</p>
<p>Scoring: 6. 2 points each for processing one of 3 tests correctly.</p>
<p>This evaluation rule is same in K&amp;R 1st, however, this sample cannot be processed in K&amp;R 1st based implementations since the constant token, 0U, uses the U suffix.</p>
<h4>n.13.7. Evaluation short-circuit in ||, &amp;&amp;, and ? :</h4>
<p>The order of evaluation is defined for the || and &amp;&amp; operators and the right side is not evaluated if the result is determined by the left side evaluation. In ? :, either the second or third is evaluated but not the other as a result of the first operand evaluation. *1</p>
<p>Therefore, even division by 0 is not an error in the term not evaluated.</p>
<p>Scoring: 6. Subtract 2 points each from 6 if one of the 5 samples fails. 0 point for failure of 3 or more samples. Subtract 2 points for wrong diagnostics, even if an implementation succeeds to process.</p>
<p>Note:</p>
<p>*1 In the ? : operator, however, the usual arithmetic conversion is performed between the second operand and the third one. It is strange to perform a conversion even when no evaluation is done. Especially, as the type for an integer constant token used in the #if expression is not determined until a value is evaluated, the value cannot be helped being evaluated in order for the type (though it is just if signed or unsigned) to be determined. However, no division by 0 is allowed. This is rather messy.</p>
<h4>n.13.8. Grouping of unary operators, -, +, !, and ~</h4>
<p>From n.13.8 to n.13.12 are tests for grouping sub-expressions in the #if expressions. Sub-expressions are grouped according to the precedence and associativity of operators. Though there are parts decided by the syntax prior to the precedence in standard integer constant expressions, the #if expressions do not have areas with syntax problems other than ( and ) of grouping. n.13.8 to n.13.10 are tests for associativity and n.13.8 is a test for the associativity for unary operators, -, +, !, and ~. All unary operators associate from right to left.</p>
<p>Scoring: 2.</p>
<h4>n.13.9. Grouping of ? :</h4>
<p>The conditional operator, ? :, is associated from right to left.</p>
<p>Scoring: 2.</p>
<h4>n.13.10. Grouping &lt;&lt; and &gt;&gt;</h4>
<p>All binary operators are associated from left to right. n.13.10 tests &lt;&lt; and &gt;&gt;.</p>
<p>Scoring: 2.</p>
<h4>n.13.11. Grouping of operators with different precedence - Part 1</h4>
<p>Here, we test expressions including unary operators, -, +, and !, and binary operators, *, /, and &gt;&gt;, which have different precedence and associativity.</p>
<p>Scoring: 2.</p>
<h4>n.13.12. Grouping of operators with different precedence - Part 2</h4>
<p>Here, we test grouping of even more complex expressions including unary operators, -, +, ~, and !, binary operators, -, *, %, &gt;&gt;, &amp;, | ^, ==, and !=, and tertiary operator, ? :.</p>
<p>Scoring: 2.</p>
<h4>n.13.13. Macros expanded into operators</h4>
<p>The use of macros are allowed in the #if expressions. These macros are usually expanded into integer constants, however, we do not test these here since they are included in n.10.1, n.12.1, n_12.2, and n.13.7 tests.</p>
<p>Though macros expanded into operators are not ordinary, they should be handled as operators in principle. A standard header called &lt;iso646.h&gt; was defined as a specification in ISO C 1990/Amendment 1 which defines some operators using macros (*1.) The purpose seems to be that source can be written without using characters such as &amp;, |, !, ^, and ~. Preprocessing is required to expand these macros in #if and handle them as operators.</p>
<p>On the other hand, there is a specification in which it is undefined if macros in the #if expressions are expanded to 'defined'. I suspect that defined is handled separately since it is similar to an identifier (refer to <a href=#u.1.19>u.1.19</a>.)</p>
<p>Scoring: 4.</p>
<p>Note:</p>
<p>*1 In C++ Standard, these identifier-like operators are not macros but tokens for some reasons.</p>
<h4>n.13.14. Macro expanded into 0 piece of token</h4>
<p>The #if expression including a macro which expands into 0 piece of token is not ordinary, either. This should be evaluated after the macro is removed (expanded.)</p>
<p>Scoring: 2.</p>
<h3><a name="3.4.14" href="#toc.3.4.14">3.4.14. #if Expression Error</a></h3>
<p>From e.14.1 to e.14.10 are tests for violations of syntax rules and violations of constraint in the #if expressions. Compiler systems must issue diagnostic messages for all source including one of these. *1</p>
<p>Note:</p>
<pre>
*1 C90 6.8.1 Conditional inclusion
C90 6.4 Constant expressions
C99 6.10.1 Conditional inclusion
C99 6.6 Constant expressions
</pre>
<h4>e.14.1. String literals</h4>
<p>As the #if expressions are integer constant expressions and pointers cannot be used, string literals cannot be used.</p>
<p>Scoring: 2.</p>
<h4>e.14.2. Operators such as =, ++, --, and .</h4>
<p>As the #if expressions are constant expressions, operators and variables with side effects cannot be used. A --B is different from A - -B and a violation of constraint.</p>
<p>Scoring: 4. 2 points if one of 4 samples cannot be correctly diagnosed. 0 point if 2 or more.</p>
<h4>e.14.3. Incomplete expressions</h4>
<p>Missing one operand in a binary operator or parenthesis is also a violation of syntax rules.</p>
<p>Scoring: 2.</p>
<h4>e.14.4. Missing #if defined parenthesis</h4>
<p>An argument for defined operator on the #if line may or may not be enclosed by ( and ), however, it is a violation of constrains if only one of the parenthesis pair exists.</p>
<p>Scoring: 2.</p>
<h4>e.14.5. No expressions</h4>
<p>Only #if without any expression is certainly a violation of syntax rules.</p>
<p>Scoring: 2.</p>
<h4>e.14.6. No expression after macro expansion</h4>
<p>The identifier not defined as a macro is evaluated as 0, however, the argument of the #if line which disappears after macro expansion is a violation of syntax rules.</p>
<p>Scoring: 2.</p>
<h4>e.14.7. Unrecognized keyword - sizeof</h4>
<p>sizeof, a pp-token, is simply treated as an identifier and evaluated as 0 in the #if expression if it is not defined as a macro. A pp-token called int is the same. Therefore, sizeof (int) becomes 0 (0) and it is a violation of syntax rules.</p>
<p>Scoring: 2.</p>
<h4>e.14.8. Unrecognized keyword - type name</h4>
<p>Just as e.14.7, (int)0x8000 becomes (0)0x8000 and it is a violation of syntax rules.</p>
<p>Scoring: 2.</p>
<h4>e.14.9. Division by 0</h4>
<p>This e.14.9 and the next e.14.10 admit of several interpretations regarding a diagnostic message should be issued and specifications are ambiguous. There are following specifications in the Standards.</p>
<p>C90 6.4 Constant expressions -- Constraints<br>
C99 6.6 Constant expressions -- Constraints</p>
<blockquote>
<p>Each constant expression shall evaluate to a constant that is in the range of representable values for its type.</p>
</blockquote>
<p>The applicable range in this specification is not clear, however, it is clear that this is applied to at least where constant expressions are necessary. The #if expressions must be constant expressions. On the other hand, there are specifications below.</p>
<p>C90 6.3.5 Multiplicative operators -- Semantics<br>
C99 6.5.5 Multiplicative operators -- Semantics</p>
<blockquote>
<p>if the value of the second operand is zero, the behavior is undefined.</p>
</blockquote>
<p>C90 6.1.2.5 Types<br>
C99 6.2.5 Types</p>
<blockquote>
<p>A computation involving unsigned operands can never overflow,</p>
</blockquote>
<p>Which specification should be applied between division by 0 and unsigned operation? It seems either interpretation is possible.</p>
<p>However, we will make an interpretation this way here -- include division by 0 where a constant expression is required and a diagnostic message must be issued in case it does not fit in the range of the type --. That is because it seems appropriate to issue a diagnostic since only an error in program can cause this type of result and constant expressions are something evaluated at compilation rather than execution. In addition, that is also because it is unnatural to treat only division by 0 as an exception. However, since the specification gets doubly vague in case the result of an unsigned operation is out of range, we do not include it here, but interpret it as undefined.</p>
<p>ISO 9899:1990/Corrigendum 1 added a specification, "A conforming implementation shall produce at least one diagnostic message .. if .. translation unit contains a violation of any syntax rule or constraint, even if the behavior is also explicitly specified as undefined or implementation-defined." This was carried on by C99. *1</p>
<p>Scoring: 2.</p>
<p>Note:</p>
<pre>
*1 C99 5.1.1.3 Diagnostics
</pre>
<h4>e.14.10. Operation results out of representable range</h4>
<p>In C90, values of the #if expressions must be in the range representable as long/unsigned long.</p>
<p>Scoring: 4. 4 points for correctly diagnosing all 4 tests. 2 points for correctly diagnosing 2 or 3 tests. 0 points if only 1 or none is correctly diagnosed.</p>
<h3><a name="3.4.15" href="#toc.3.4.15">3.4.15. #ifdef and #ifndef</a></h3>
<p>The n.15.* test #ifdef and #ifndef. These are exactly same between K&amp;R 1st and Standard C. e.15 tests the violation of syntax rules for that. *1</p>
<p>Note:</p>
<pre>
*1 C90 6.8 Preprocessing directives -- Syntax
C90 6.8.1 Conditional inclusion
C99 6.10 Preprocessing directives -- Syntax
C99 6.10.1 Conditional inclusion
</pre>
<h4>n.15.1. #ifdef macro testing</h4>
<p>Scoring: 6.</p>
<h4>n.15.2. #ifndef macro testing</h4>
<p>Scoring: 6.</p>
<h4>e.15.3. Argument which is not an identifier</h4>
<p>Arguments on the #ifdef and #ifndef lines must be identifiers.</p>
<p>Scoring: 2.</p>
<h4>e.15.4. Extra argument</h4>
<p>Arguments on the #ifdef and #ifndef lines must not have extra tokens other than identifiers.</p>
<p>Scoring: 2.</p>
<h4>e.15.5. Missing argument</h4>
<p>It is a violation of syntax rules not to have any arguments.</p>
<p>Scoring: 2.</p>
<h3><a name="3.4.16" href="#toc.3.4.16">3.4.16. #else and #endif Errors</a></h3>
<p>Next is a test of violations of syntax rules for #else and #endif. This syntax has not changed since K&amp;R 1st. (However, Standard C introduced a new specification that a diagnostic message must be issued for a violation of syntax rules or constraints.) *1</p>
<p>Note:</p>
<pre>
*1 C90 6.8 Preprocessing directives -- Syntax
C99 6.10 Preprocessing directives -- Syntax
</pre>
<h4>e.16.1. Extra token in #else</h4>
<p>The #else line must not have any other tokens.</p>
<p>Scoring: 2.</p>
<h4>e.16.2. Extra token in #endif</h4>
<p>The #endif line must not have any other tokens.</p>
<p>Do not write below.</p>
<pre style="color:navy">
#if MACRO
#else ! MACRO
#endif MACRO
</pre>
<p>Use below instead.</p>
<pre style="color:navy">
#if MACRO
#else /* ! MACRO */
#endif /* MACRO */
</pre>
<p>Scoring: 2.</p>
<h3><a name="3.4.17" href="#toc.3.4.17">3.4.17. #if, #elif, #else, and #endif Mismatching Errors</a></h3>
<p>Next tests violations of syntax rules for matching #if (#ifdef, #ifndef), #elif, #else, and #endif. This syntax is almost the same since K&amp;R 1st except that #elif was added to Standard C. In addition, K&amp;R 1st was not clear in the point that these must match within the source file unit. *1</p>
<p>Note:</p>
<pre>
*1 C90 6.8 Preprocessing directives -- Syntax
C99 6.10 Preprocessing directives -- Syntax
</pre>
<h4>e.17.1. #endif without #if</h4>
<p>#endif without a preceding #if is obviously a violation of syntax rule.</p>
<p>Scoring: 2.</p>
<h4>e.17.2. #else without #if</h4>
<p>#else without a corresponding #if is also an error.</p>
<p>Scoring: 2.</p>
<h4>e.17.3. Another #else after #else</h4>
<p>Having another #else after a #else is also prohibited.</p>
<p>Scoring: 2.</p>
<h4>e.17.4. #elif after #else</h4>
<p>#elif after #else is not allowed.</p>
<p>Scoring: 2.</p>
<h4>e.17.5. #endif without #if in the file included</h4>
<p>#if, #else, and #endif must be matched in the source file (preprocessing file) unit. It is not acceptable for the file included to be treated as if it existed from the beginning or in the original file.</p>
<p>Scoring: 2.</p>
<h4>e.17.6. #if without #endif in the file included</h4>
<p>Scoring: 2.</p>
<h4>e.17.7. #if without #endif</h4>
<p>Forgetting #endif actually happens quite often, however, compiler systems must issue a diagnostic message for that.</p>
<p>Scoring: 2.</p>
<h3><a name="3.4.18" href="#toc.3.4.18">3.4.18. #define</a></h3>
<p>For the #define syntax, the # and ## operators were added in C90 whereas they did not exit in K&amp;R. The rest is unchanged. *1</p>
<p>In C99, variable argument macros were added (refer to <a href="#2.8">2.8</a>.) *2</p>
<p>Note:</p>
<pre>
*1 C90 6.8 Preprocessing directives -- Syntax
C90 6.8.3 Macro replacement
*2 C99 6.10 Preprocessing directives -- Syntax
C99 6.10.3 Macro replacement
</pre>
<h4>n.18.1. Object-like macro definition</h4>
<p>The first token on the #define line is a macro name. However, in case there are white spaces immediately after, the second token is considered to be the beginning of the replacement list even if it is '(' and not considered as a function-like macro definition. If there is no token after a macro name, the macro is defined as 0 pieces of token.</p>
<p>Scoring: 30. 10 points if only one of 2 macros are defined correctly.</p>
<h4>n.18.2. Function-like macro definition</h4>
<p>If '(' is immediately after a macro name without having white spaces between, it is considered to be the beginning of the function-like macro parameter list. This specification has been around since K&amp;R 1st and has a trace of character-oriented preprocessing which is influenced by the existence of white spaces. Nothing can be done at this point.</p>
<p>Scoring: 20.</p>
<h4>n.18.3. No replacement applicable inside string literals</h4>
<p>In a so-called "Reiser" model preprocessor, the same spelling as a parameter in string literals or character constants in the replacement list, that portion was substituted for an argument by macro expansion. However, this was not accepted by Standard C nor K&amp;R 1st. This replacement is a specification characterized well by character-oriented preprocessing, however, it is out of the question in token-oriented processing.</p>
<p>Scoring: 10.</p>
<h4>n.vargs. Variable argument macros</h4>
<p>Variable argument macros were introduced in C99.</p>
<p>Scoring: 10.</p>
<h4>e.18.4. Missing name</h4>
<p>The first token on the #define line must be an identifier.</p>
<p>Scoring: 2.</p>
<h4>e.18.5. Missing argument</h4>
<p>If there is not a single token on the #define line, it is a violation of syntax rules.</p>
<p>Scoring: 2.</p>
<h4>e.18.6. Empty parameter</h4>
<p>Empty parameter is also a violation of syntax rules. *1</p>
<p>Scoring: 2.</p>
<p>Note:</p>
<pre>
*1 C90 6.5.4 Declarators -- Syntax
C99 6.7.5 Declarators -- Syntax
</pre>
<h4>e.18.7. Duplicate parameter names</h4>
<p>Duplicate parameter names in the parameter list for one macro definition is a violation of constraints. *1</p>
<p>Scoring: 2.</p>
<p>Note:</p>
<pre>
*1 C90 6.8.3 Macro replacement -- Constraints
C99 6.10.3 Macro replacement -- Constraints
</pre>
<h4>e.18.8. Parameter which is not an identifier</h4>
<p>A parameter in a macro definition must be an identifier. *1</p>
<p>Scoring: 2.</p>
<p>Note:</p>
<p>*1 The ... parameter was added in C99. __VA_ARGS__ in the replacement list is a special parameter name that corresponds to it.</p>
<h4>e.18.9. Special combination of a macro name and a replacement list</h4>
<p>Though '$' is not accepted as a character within an identifier in Standard C, there are compiler systems which accept it. This sample is the kind that can be seen in the source compiled by those compiler systems. Since this example interprets $ as one character and one pp-token in Standard C, the macro name is THIS and $ and after becomes a replacement list for an object-like macro which is totally different from the program purpose of a function-like macro named THIS$AND$THAT.</p>
<p>In the Corrigendum 1 of ISO 9899:1990, an exceptional specification was added to this type of example. A conforming implementation must issue a diagnostic message regarding this example. *1</p>
<p>Conversely, C99 defined that white spaces must exist between a macro name a replacement list in object-like macro definition in general. *2</p>
<p>Scoring: 2.</p>
<p>Note:</p>
<p>*1 Addition to Constraints in C90 6.8 by Corrigendum 1.<br>
This specification, however, has disappeared in C++ Standard.</p>
<pre>
*2 C99 6.10.3 Macro replacement -- Constraints
</pre>
<h4>e.vargs1. __VA_ARGS__ not in the replacement list</h4>
<p>Variable argument macros in C99 use __VA_ARGS__ in the replacement list corresponding to a parameter ... in the parameter list of the macro definition. This identifier must not be used anywhere else.</p>
<p>Scoring: 2. 2 points if 2 samples are correctly diagnosed. 0 points if only one is diagnosed correctly.</p>
<h3><a name="3.4.19" href="#toc.3.4.19">3.4.19. Macro Re-definition</a></h3>
<p>Macro re-definition was not mentioned in K&amp;R 1st and implementations were all different as well. In Standard C, some re-definition of the original definition is allowed, but not a different one. Macros are virtually not re-defined (unless they are voided by #undef). *1, *2</p>
<p>Note:</p>
<pre>
*1 C90 6.8.3 Macro replacement -- Constraints
C99 6.10.3 Macro replacement -- Constraints
</pre>
*2 However, many compiler systems issue a warning and accept redefinition. So does <b>mcpp</b> starting with V.2.4, to maintain compatibility with existing compiler systems.</p>
<h4>n.19.1. Differences in white spaces before and after a replacement list</h4>
<p>Re-definition where only the number of white spaces are different, is allowed.</p>
<p>Scoring: 4.</p>
<h4>n.19.2. Differences in white spaces in a parameter list and differences in white spaces extending over lines</h4>
<p>White spaces include ones extending over source lines by &lt;backslash&gt;&lt;newline&gt; sequence or comments.</p>
<p>Scoring: 4.</p>
<h4>e.19.3. Differences in token sequence in a replacement list</h4>
<p>Re-definition where token sequences in a replacement list are different is a violation of constraint.</p>
<p>Scoring: 4.</p>
<h4>e.19.4. Differences in the existence of white spaces in a replacement list</h4>
<p>Re-definition where the existence of white spaces in a replacement list is different is a violation of constraints. This has a trace of character-oriented preprocessing.</p>
<p>Scoring: 4.</p>
<h4>e.19.5. Different parameter usage</h4>
<p>As re-definition of different parameter usage is essentially a different definition, it is a violation of constraints.</p>
<p>Scoring: 4.</p>
<h4>e.19.6. Differences in parameter names</h4>
<p>Re-definition where only parameter names are different but which is essentially the same is a violation of constraints. This seems to be an excessive constraint.</p>
<p>Scoring: 2.</p>
<h4>e.19.7. Sharing function-like and object-like macro names</h4>
<p>As a macro name belongs to one name space, a function-like macro and an object-like macro cannot use the same name.</p>
<p>Scoring: 2.</p>
<h3><a name="3.4.20" href="#toc.3.4.20">3.4.20. Macro Names Same as Keywords</a></h3>
<p>Since no keyword exists in preprocessing, an identifier with same name as a keyword can be defined as a macro and expanded. *1</p>
<p>Note:</p>
<pre>
*1 C90 6.1 Lexical elements -- Syntax
C99 6.4 Lexical elements -- Syntax
C89 Rationale 3.8.3 (C99 Rationale 6.10.3) Macro replacement
</pre>
<h4>n.20.1. Names are all subject to macro expansion</h4>
<p>Scoring: 6.</p>
<h3><a name="3.4.21" href="#toc.3.4.21">3.4.21. Macro Expansion Requiring Pp-token Separation</a></h3>
<p>Tokenizing a source file into pp-tokens is performed in translation phase 3. And the case where multiple pp-tokens are concatenated into 1 pp-token later is defined only by the case where the concatenation is done by expanding the macro defined using the ## operator, the case where stringizing takes place by macro expansion defined by the # operator, and the case of concatenating adjacent string literals. Therefore, it is interpreted that implicit concatenation of multiple pp-tokens must not happen. This is obvious in the principle of token-oriented preprocessing. *1</p>
<p>Note:</p>
<pre>
*1 C90 5.1.1.2 Translation phases
C90 6.8.3 Macro replacement
C99 5.1.1.2 Translation phases
C99 6.10.3 Macro replacement
</pre>
<h4>n.21.1. Pp-tokens are not concatenated implicitly</h4>
<p>In case preprocessing is done in an independent program, it is necessary to separate and pass 3 -'s in the output of this sample by some sort of token separator so that compiler proper can figure out 3 are separate pp-tokens.</p>
<p>Scoring: 4.</p>
<h4>n.21.2. Separation of macro in outer macro's argument</h4>
<p>Even if a macro is invoked in an argument of outer macro, expansion result of the macro should not be merged with its surrounding pp-tokens in outer macro's replacement list.</p>
<p>Scoring: 2.</p>
<h3><a name="3.4.22" href="#toc.3.4.22">3.4.22. Macro-like Sequence in a Pp-number</a></h3>
<p>Preprocessing-numbers were introduced by Standard C for the first time. They have a wider range than integer constant tokens and floating point constant tokens put together and may include identifier-like portion. They were defined as a specification in order to simplify tokenization in preprocessing. However, in case there is a macro-like sequence in a pp-number, a wrong result may occur unless this simple tokenization is done exactly. *1</p>
<p>Note:</p>
<pre>
*1 C90 6.1.8 Preprocessing numbers
C99 6.4.8 Preprocessing numbers
</pre>
<h4>n.22.1. Macro-like Sequence in a Pp-number - Part 1</h4>
<p>Since the sequence, 12E+EXP, is one pp-number, it will not be expanded even if a macro, EXP, is defined.</p>
<p>Scoring: 4.</p>
<h4>n.22.2. Macro-like Sequence in a Pp-number - Part 2</h4>
<p>A pp-number starts with a digit or .</p>
<p>Scoring: 2.</p>
<h4>n.22.3. Macros outside of a pp-number are expanded</h4>
<p>In C90, + or - can appear inside a pp-number only if it immediately follows E or e. 12+EXP is different from 12E+EXP and is divided into 3 pp-tokens, 12, +, and EXP. These are a pp-number, an operator, and an identifier, respectively. EXP is expanded if it is a macro.</p>
<p>Scoring: 2.</p>
<h4>n.ppnum. [Pp][+-] sequence</h4>
<p>In C99, the sequence of + or - following P or p in a pp-number was added to write a floating point number in hexadecimal.</p>
<p>In order to display a floating point number in printf(), conversion specifiers such as %a and %A are used. *1</p>
<p>Scoring: 4.</p>
<p>Note:</p>
<pre>
*1 C99 7.19. 6.1 The fprintf function -- Description
</pre>
<h3><a name="3.4.23" href="#toc.3.4.23">3.4.23. Macros Using the ## Operator</a></h3>
<p>## is an operator newly introduced in Standard C and used only in the replacement list on the #define line. The pp-tokens before and after a ## are concatenated into one pp-token. If pp-tokens before and after ## are parameters, they are first replaced by actual arguments at macro expansion and concatenated. *1</p>
<p>Note:</p>
<pre>
*1 C90 6.8.3.3 The ## operator
C99 6.10.3.3 The ## operator
</pre>
<h4>n.23.1. Token concatenation</h4>
<p>This is an example of the most simple function-like macro using the ## operator.</p>
<p>Scoring: 6.</p>
<h4>n.23.2. Pp-number generation</h4>
<p>Since operands of the ## operator are not macro-expanded, using another macro which appears to be meaningless together such as xglue() in this example often takes place. This is for expanding a macro in an argument, then concatenating the result. The 12e+2 sequence which was generated by a macro call in this sample is a valid pp-number.</p>
<p>Scoring: 2.</p>
<h4>e.23.3. Missing tokens before or after ## - object-like macros</h4>
<p>There must be some pp-tokens before and after the ## operator in a replacement list. This is an example of an object-like macro. It is meaningless to use ## in an object-like macro, but not an error.</p>
<p>Scoring: 2.</p>
<h4>e.23.4. Missing tokens before or after ## - function-like macros</h4>
<p>This is an example of a function-like macro definition without pp-tokens before or after the ## operator in a replacement list.</p>
<p>Scoring: 2.</p>
<h3><a name="3.4.24" href="#toc.3.4.24">3.4.24. Macros Using the # Operator</a></h3>
<p>The # operator was introduced in Standard C. It is used only in the replacement list for the #define line which defines a function-like macro. The operands of the # operator are parameters and corresponding actual arguments at those macro expansion are converted to string literals. *1</p>
<p>Note:</p>
<pre>
*1 C90 6.8.3.2 The # operator
C99 6.10.3.2 The # operator
</pre>
<h4>n.24.1. Argument stringizing</h4>
<p>The argument corresponding to the operand for the # operator is enclosed by " and " on both ends and stringized.</p>
<p>Scoring: 6. 2 points if a space is inserted between token as "a + b".</p>
<h4>n.24.2. White space handling between argument tokens</h4>
<p>In case the argument corresponding to the operand for the # operator comprise of multiple pp-token sequence, white spaces between those pp-tokens are converted into a space and stringized. No space is inserted if there are no white spaces. That is to say, the results differ by the existence of white spaces though they are not influenced by the number of white spaces (this still has a trace of character-oriented preprocessing.) White spaces before and after an argument are deleted.</p>
<p>Scoring: 4.</p>
<h4>n.24.3. \ insertion</h4>
<p>In case a string literal or character constant is in the argument corresponding to the operand for the # operator, \ is inserted immediately prior to the \ or " in it and the " which encloses a string literal. This is same as the method of writing string literals to display string literals or character constants as they are.</p>
<p>Scoring: 6.</p>
<h4>n.24.4. Calling macro including &lt;backslash&gt;&lt;newline&gt;</h4>
<p>As the &lt;backslash&gt;&lt;newline&gt; sequences are removed in translation phase 2, they do not exist at macro expansion.</p>
<p>Scoring: 2.</p>
<h4>n.24.5. Token separator inserted by macro expansion should not remain</h4>
<p>Macro expansion routine generally guards the expansion result with token separators in order to avoid token-merging with surrounding tokens. (See <a href="#3.4.21">3.4.21</a>.) In case of stringization, however, the inserted token-separators should not remain.</p>
<p>These cumbersome issues arise from character-oriented portion of the Standard mixed into token-based principle.</p>
<p>Scoring: 2.</p>
<h4>e.24.6. Operand for the # operator is not a parameter name</h4>
<p>The operand for the # operator must be a parameter name.</p>
<p>Scoring: 2.</p>
<h3><a name="3.4.25" href="#toc.3.4.25">3.4.25. Macro Expansion in a Macro Argument</a></h3>
<p>When macros in an argument should be expanded in a function-like macro call was not mentioned in K&amp;R 1st. Implementations were all different in pre-C90 preprocessors. More might have expanded at rescanning a replacement list. In Standard C, it was defined as a specification that this was expanded after an argument was identified first, then before substituted for a parameter. The order is similar to the argument evaluation of function calls, which is easy to understand. However, in case the argument corresponds the parameter which is an operand for the # or ## operator, a macro is not considered as a macro call and is not get expanded even if the macro name is included. *1</p>
<p>Note:</p>
<pre>
*1 C90 6.8.3.1 Argument substitution
C99 6.10.3.1 Argument substitution
</pre>
<h4>n.25.1. Macro in an argument is expanded first</h4>
<p>A macro in an argument is expanded after the argument is identified, then substituted for a parameter in the replacement list. Therefore, the one identified as one argument is one argument even if it appears multiple arguments after expansion.</p>
<p>Scoring: 4.</p>
<h4>n.25.2. Argument expanded into 0 piece of token</h4>
<p>Similarly, arguments which become 0 piece of token after expansion are legitimate.</p>
<p>Scoring: 2.</p>
<h4>n.25.3. Calling macro using the ## operator in an argument</h4>
<p>In case the operand for the ## operator is a parameter, the argument corresponding to it does not get macro-expanded. Therefore, it is necessary to nest another macro in order to do macro expansion.</p>
<p>Since xglue() does not use the ## operator in this example, glue( a, b), which is an argument for it, gets macro-expanded and becomes 'ab' and the replacement for xglue() becomes glue(ab, c). This is rescanned and 'abc' is the final result.</p>
<p>Scoring: 2.</p>
<h4>n.25.4. No expansion for the operand for the ## operator</h4>
<p>Since glue() is directly called, a macro does not get expanded even if a macro name is in the argument.</p>
<p>Scoring: 6.</p>
<h4>n.25.5. No expansion for the operand for the # operator</h4>
<p>The argument corresponding to the parameter which is an operand for the # operator is not macro-expanded.</p>
<p>Scoring: 4.</p>
<h4>e.25.6. Macro expansion in an argument incomplete in the argument</h4>
<p>Macro expansion in an argument is done only in that argument. Incomplete macro is a violation of constraints. Though a function-like macro name is not a macro call by itself, it becomes the beginning of a macro call sequence if it is followed by '('. Once it begins, ')' corresponding to this '(' must exist. *1</p>
<p>Scoring: 4.</p>
<p>Note:</p>
<pre>
*1 C90 6.8.3 Macro replacement -- Constraints
C99 6.10.3 Macro replacement -- Constraints
</pre>
<h3><a name="3.4.26" href="#toc.3.4.26">3.4.26. Macros of a Same Name during Macro Rescanning</a></h3>
<p>In case macro definition is recursive, expanding the macro as is causes infinite recursion. Because of this, recursive macros could not be used in K&amp;R 1st. Standard C introduced a new specification that the macro name replaced once does not get replaced again in order to allow recursive macro expansion, preventing infinite recursion. This specification is quite difficult, but can be paraphrased as below. *1</p>
<ol>
<li>While rescanning the replacement list of macro A, the name, A, even if found, will not be replaced again.<br>
<br>
<li>In case of nested replacement which has the name, A, in the middle of macro B replacement when there is a macro B call in the replacement list of macro A, the A will not be replaced again.<br>
<br>
<li>However, in case of macro B replacement included the token sequence after the replacement list of the original macro A, A at this part included is replaced, if and only this A is in the source file (not in replacement list of any macro).<br>
<br>
<li>1 to 3 are recursively applied to macro B replacement. In other words, if B appears again in the middle of macro B replacement and it is on the token sequence after the replacement list of the original B, and this B is not in the source file, this B will not be replaced.<br>
<br>
<li>In case there is macro C call in the argument for the original macro A call and C which appeared in the middle of replacement of C applies 1 through 3 and is forbidden from replacement, this C will not be replaced again even when the replacement list of the original macro A is rescanned.<br>
</ol>
<p>After all these paraphrasing, it is still difficult. Especially, 3 comes from a traditional macro rescan specification of including subsequent token sequence and this complicates the problem unnecessarily. The Standard has issued a corrigendum and made corrections concerning this subject, however, it just gets more confusing. Furthermore, the Standard changes macro expansion depending whether the subsequent token sequence is in the source file or not. This is an inconsistent specification. *2</p>
<p>Not only that the macro expansion involving the succeeding token sequence is uncommon, but also it is doubly uncommon that the macro, whose re-replacement at that part has been prohibited, appears again. The Validation Suite does not have this type of macro as a test item other than n.27.6. I hope this abnormal specification regarding macro expansion of "including a succeeding token sequence" will be removed. *3</p>
<p>Note:</p>
<pre>
*1 C90 6.8.3.4 Rescanning and further replacement
C99 6.10.3.4 Rescanning and further replacement
</pre>
<p>*2 Refer to <a href="#2.7.6">2.7.6</a>.</p>
<p>*3 At the newsgroup comp.std.c, there has been some controversy on the Standard's specification about recursive macro expansion. Mainly two interpretations have been insisted on this subject. recurs.t is one of the samples used in the discussion. Refer to the comment in recurs.t. This Validation Suite does not evaluate preprocessors behavior on this sample.</p>
<p><b>mcpp</b> V.2.4.1 or later in Standard mode implements the two ways of recursive macro expansion. <b>mcpp</b> sets, by default, the range of non-re-replacing of the same name as wide as the explanation above 1-5, and expands this sample as 'NIL(42)'. <b>mcpp</b> sets, by '-@compat' option, the range narrower, and expands this sample as '42'. The difference of these two specifications appears, in 3 above, when the first half of a function-like macro call of A is in the replacement list of B. <b>mcpp</b>, by default, prohibit re-replacing of A even if only the name of A is in the replacement list of B. On the other hand, <b>mcpp</b>, by -@compat option, prohibit re-replacing if and only the name of A and the arguments list with '(', ')' surrounding it are all in the replacement list of B, and does not distinguish whether the name is in the source file or not.</p>
<h4>n.26.1. No re-expansion for direct recursive object-like macros</h4>
<p>This is an example of direct recursion for object-like macros.</p>
<p>Scoring: 2.</p>
<h4>n.26.2. No re-expansion for indirect recursive object-like macros</h4>
<p>This is an example of indirect recursion for object-like macros.</p>
<p>Scoring: 2.</p>
<h4>n.26.3. No re-expansion for direct recursive function-like macros</h4>
<p>This is an example of direct recursion for function-like macros.</p>
<p>Scoring: 2.</p>
<h4>n.26.4. No re-expansion for indirect recursive function-like macros</h4>
<p>This is an example of indirect recursion for function-like macros.</p>
<p>Scoring: 2.</p>
<h4>n.26.5. Recursive macros in arguments</h4>
<p>In Standard C, there is a difficult specification meaning "the macro whose re-replacement has been prohibited will not be replaced at rescan in another context." What applies to this in concrete is the handling at rescanning the parent macro for the macro in the argument. When there is a recursive macro in an argument, it is replaced only once. It is not replaced at the rescan of the parent macro, either.</p>
<p>Scoring: 2.</p>
<h3><a name="3.4.27" href="#toc.3.4.27">3.4.27. Macro Rescanning</a></h3>
<p>Rescanning of macro replacement lists has been a specification since K&amp;R 1st. Macros found at rescan are replaced as long as they are not recursive macros. This takes care of nested macro definitions. There was no special change in Standard C, though what was not obvious in K&amp;R 1st became somewhat clearer. *1</p>
<p>Note:</p>
<pre>
*1 C90 6.8.3.4 Rescanning and further replacement
C99 6.10.3.4 Rescanning and further replacement
</pre>
<h4>n.27.1. Nested object-like macro</h4>
<p>This is same in K&amp;R 1st as well.</p>
<p>Scoring: 6.</p>
<h4>n.27.2. Nested function-like macro</h4>
<p>This is same in K&amp;R 1st as well.</p>
<p>Scoring: 4.</p>
<h4>n.27.3. Name generated by the ## operator is subject to expansion</h4>
<p>The argument corresponding to the operand for the ## operator is not macro-expanded, however, the pp-token newly generated by pp-token concatenation is subject to macro expansion at rescan.</p>
<p>Scoring: 2.</p>
<h4>n.27.4. Function-like macros formed in replacement list</h4>
<p>In case a function-like macro name, even if any, does not follow (, it is not considered to be a macro call. If a function-like macro name is obtained from an argument and a function-like macro call is formed by using the name in the replacement list, it will be expanded.</p>
<p>Scoring: 6.</p>
<h4>n.27.5. Replacement list forming the first half of a function-like macro</h4>
<p>The unusual macro that a replacement list forms the first half of a function-like macro call was an unspoken specification in pre-Standard, but was officially accepted in Standard C. The pp-token sequence subsequent to the replacement list is included to complete a macro call.</p>
<p>Scoring: 2.</p>
<h4>n.27.6. Same name macro to be re-replaced</h4>
<p>In general, the same name macro is not re-replaced while rescanning. Some unusual cases are, however, re-replaced. In case of nested macro call, when the replacement list involves the subsequent pp-tokens and finds the same name in source file, the name is re-replaced.</p>
<p>Scoring: 2.</p>
<h4>e.27.7. Mismatched number of arguments at rescan</h4>
<p>The number of arguments must match that of parameters in function-like macro calls. It is also the same in the function-like macros found at the rescan of the replacement list. However, the number of arguments may not be easy to recognize intuitively in tricky macros since arguments are separated by ,.</p>
<p>Scoring: 2.</p>
<h3><a name="3.4.28" href="#toc.3.4.28">3.4.28. Predefined Macros</a></h3>
<p>C90 introduced a specification that 5 special macros are predefined by implementations. *1, *2</p>
<p>Furthermore, a macro named <tt>__STDC_VERSION__</tt> was introduced in C90/Amendment 1.</p>
<p><tt>__FILE__</tt> and <tt>__LINE__</tt> are extremely special macros that change definitions dynamically. It is used in the assert(), debug tools, and others. The rest of Standard predefined macros do not change during the processing of one translation unit.</p>
<p>Note:</p>
<pre>
*1 C90 6.8.8 Predefined macro names
*2 C99 6.10.8 Predefined macro names
</pre>
<h4>n.28.1. __FILE__</h4>
<p>This is defined to be the string literal for the source file name being preprocessed. As the file name is used for the source file included by #include, it changes in the same translation unit also.</p>
<p>Scoring: 4. 0 points for just a name as in n_28.t, rather than a string literal as in "n_28.t".</p>
<h4>n.28.2. __LINE__</h4>
<p>This is defined to the decimal constant for the line number in the source file being preprocessed. The line number starts with 1.</p>
<p>Scoring: 4. 2 points if the line number starts with 0.</p>
<h4>n.28.3. __DATE__</h4>
<p>This is defined to the string literal for the date on which preprocessing is performed. The format is "Mmm dd yyyy" and almost the same as the one generated by the asctime() function with an exception that the 1st digit of dd is a space, not 0, in case the day is prior to the 10th.</p>
<p>Scoring: 4. 2 points if the 1st digit of dd is 0 in case the day is prior to the 10th.</p>
<h4>n.28.4. __TIME__</h4>
<p>This is defined to the string literal for the time at which preprocessing is performed. The format is "hh:mm:ss", same as the one generated by the asctime() function.</p>
<p>Scoring: 4.</p>
<h4>n.28.5. __STDC__</h4>
<p>This is defined to a constant, 1, in C90 or C99 compliant implementations.</p>
<p>Scoring: 4.</p>
<h4>n.28.6. __STDC_VERSION__</h4>
<p>This is defined to 199409L in implementations supporting C90/Amendment 1:1995. *1</p>
<p>Scoring: 4.</p>
<p>Note:</p>
<pre>
*1 Amendment 1/3.3 Version macro (addition to ISO 9899:1990 / 6.8.8)
</pre>
<h4>n.stdmac. C99 predefined macros</h4>
<p>In C99, the value of <tt>__STDC_VERSION__</tt> is 199901L.</p>
<p>Also, a predefined macro, <tt>__STDC_HOSTED__</tt>, was added. This is defined to 1 for a hosted implementation, 0 otherwise.</p>
<p>Scoring: 4. 2 points each.</p>
<h4>n.28.7. __LINE__ and others in files included</h4>
<p>Since <tt>__FILE__</tt> and <tt>__LINE__</tt> are subject to the source files, not translation units, they are the name of file included and a line number in the source included.</p>
<p>Scoring: 4. 2 points if the line number starts with 0.</p>
<h3><a name="3.4.29" href="#toc.3.4.29">3.4.29. #undef</a></h3>
<p>#undef has been supported since K&amp;R 1st and there are no major changes. It cancels the macro definition previously defined. The valid range for the macro is from when it is defined by #define until when it is canceled by #undef. *1</p>
<p>Note:</p>
<pre>
*1 C90 6.8.3.5 Scope of macro definitions
C99 6.10.3.5 Scope of macro definitions
</pre>
<h4>n.29.1. Macro cancellation by #undef</h4>
<p>The macro name is no longer a macro after #undef.</p>
<p>Scoring: 10.</p>
<h4>n.29.2. #undef for the macro without being defined</h4>
<p>Applying #undef to the name which has not been defined as a macro is allowed. Compiler systems must not reject this as an error.</p>
<p>Scoring: 4.</p>
<h4>e.29.3. Missing name</h4>
<p>An identifier is required on the #undef line.</p>
<p>Scoring: 2.</p>
<h4>e.29.4. Extra token</h4>
<p>The #undef line must not have other than one identifier.</p>
<p>Scoring: 2.</p>
<h4>e.29. 5. Missing argument</h4>
<p>Missing an argument on the #undef line is a violation of syntax rules.</p>
<p>Scoring: 2.</p>
<h3><a name="3.4.30" href="#toc.3.4.30">3.4.30. Macro Calls</a></h3>
<p>In a macro call, a &lt;newline&gt; is treated as simply one of white spaces. Therefore, a macro call can extend over multiple lines, which was not clear in K&amp;R 1st. *1</p>
<p>Arguments for function-like macro calls are separated by ','. However, the , inside of a pair of ( and ) is not considered to be a separator for arguments. This is not tested here directly, however, it is implicitly tested throughout the Suite, in n.25 as an example. Since many of *.c samples use the assert() macro, quite complicated testing regarding this is performed.</p>
<p>Note:</p>
<pre>
*1 C90 6.8.3 Macro replacement -- Semantics
C99 6.10.3 Macro replacement -- Semantics
</pre>
<h4>n.30.1. Macro call extending over multiple lines</h4>
<p>Scoring: 6.</p>
<h4>n.nularg. Empty arguments in macro calls</h4>
<p>In C99, an empty argument in macro calls was accepted. This is different from insufficient arguments. The ',' to separate arguments cannot be omitted (though both cannot be identified in case of one parameter.)</p>
<p>Scoring: 6.</p>
<h3><a name="3.4.31" href="#toc.3.4.31">3.4.31. Macro Call Error</a></h3>
<p>The next are some errors regarding macro calls.</p>
<h4>e.31.1. Too many arguments</h4>
<p>Different number of arguments and parameters is a violation of constraint, not undefined. *1</p>
<p>Scoring: 2.</p>
<p>Note:</p>
<pre>
*1 C90 6.8.3 Macro replacement -- Constraints
C99 6.10.3 Macro replacement -- Constraints
</pre>
<h4>e.31.2. Insufficient arguments</h4>
<p>The number of arguments less than that of parameters is a violation of constraints.</p>
<p>In C99, an empty argument was accepted. This is different from insufficient arguments. The ',' to separate arguments must exist.</p>
<p>Scoring: 2.</p>
<h4>e.31.3. Incomplete macro call on the control line</h4>
<p>In general, a macro call can extend over multiple lines. However, since the preprocessing directive line starting with # completes in the line (possibly spliced by multi-line comment), a macro call in it must complete in the line as well.</p>
<p>Scoring: 2.</p>
<h4>e.vargs2. No Arguments for Variable Argument Macros</h4>
<p>Variable argument macros in C99 need at least one actual argument for __VA_ARGS__.</p>
<p>Scoring: 2.</p>
<h3><a name="3.4.32" href="#toc.3.4.32">3.4.32. Character Constant in #if Expression</a></h3>
<p>The #if expressions can use character constants. However, the evaluation is mostly implementation-defined and does not have much portability. 32.? Covers the most simple single byte character constants. *1</p>
<p>Note:</p>
<pre>
*1 C90 6.1.3.4 Character constants
C90 6.8.1 Conditional inclusion -- Semantics
C99 6.4.4.4 Character constants
C99 6.10.1 Conditional inclusion -- Semantics
</pre>
<p>Below, the sources of the Standards for the 33, 34, and 35 tests are also the same.</p>
<h4>n.32.1. Character octal escape sequences</h4>
<p>In character constants, octal escape sequences are supported. These are same in any implementation and there are no implementation-defined areas.</p>
<p>Scoring: 2.</p>
<h4>n.32.2. Character hexadecimal escape sequences</h4>
<p>In character constants, hexadecimal escape sequences are also supported. There are no implementation-defined areas here either. Hexadecimal escape sequences did not exist in K&amp;R 1st.</p>
<p>Scoring: 2.</p>
<h4>i.32.3. Single-byte character constants</h4>
<p>Single byte character constants not in an escape sequence are simple, however, values depend on basic character sets. In a cross compiler whose basic character set varies at compilation and at runtime, it is acceptable to use either of them in the #if expression evaluation.</p>
<p>Even the same basic character sets are implementation-defined in terms of sign handling. Moreover, handling can be different between the compiler proper (translation phase 7) and preprocessing (phase 4.)</p>
<p>Therefore, judging a basic character set from the value of a character constant in the #if expression is not a guaranteed method.</p>
<p>Scoring: 2.</p>
<h4>i.32.4. '\a' and '\v'</h4>
<p>Standard C added new escape sequences, '\a' and '\v'.</p>
<p>Scoring: 2.</p>
<h4>e.32.5. Escape sequence value out of the unsigned char range</h4>
<p>As an escape sequence in a character constant represents a single-byte character value, it must be in the unsigned char range.</p>
<p>Scoring: 2.</p>
<h3><a name="3.4.33" href="#toc.3.4.33">3.4.33. Wide Character Constant in #if Expression</a></h3>
<p>Wide character constants were introduced in Standard C. The value evaluation is even more implementation-defined than a single-byte character constant and even byte evaluation order is not specified.</p>
<p>Although there are various encodings of wide character, only the wide character corresponding to ASCII character is tested here. For the other encodings, see <a href="#4.1">4.1</a>.</p>
<h4>e.33.2. Wide character constant value out of range</h4>
<p>Hexadecimal or octal escape sequences can be used in wide character constant values as well, however, the values must be in the range representing one wide character value unsigned.</p>
<p>Scoring: 2.</p>
<h3><a name="3.4.35" href="#toc.3.4.35">3.4.35. Multi-Character Character Constant in #if Expressions</a></h3>
<p>Character constants include something called multi-character character constants. They appear similar to multi-byte character constants and confusing, however they are different concept and means character constants comprising of multiple characters. Among these characters, there are single-byte characters, multi-byte characters, and wide characters and the multi-character character constants corresponding to each exist (in Standards, the term, character, is used as a single-byte character, but it refers to 3 kinds of characters here.)</p>
<p>There seems to be no usage for multi-character character constants. The reason why these has been accepted since K&amp;R 1st seems to be simply because whatever is in the int range is fine as the character constant type is int.</p>
<h4>i.35.1. Multi-character character constant evaluation</h4>
<p>Multi-character character constants for single-byte characters have been around since K&amp;R 1st. (A.16.) However, the evaluation byte order is not defined in K&amp;R 1st or Standard C.</p>
<p>Scoring: 2.</p>
<h4>e.35.2. Multi-character character constant value out of range</h4>
<p>Multi-character character constant values in a hexadecimal or octal escape sequence must be in the int or unsigned int range. However, since int/unsigned int are treated as if they have the same internal representation as long/unsigned long in the #if expressions in C90, checking if values are in the long or unsigned long range seems enough. This point is not clear in the Standards. It can be interpreted that range checking is implementation-defined since the method of value evaluation itself is implementation-defined.</p>
<p>In any case, it is appropriate for this sample to issue a diagnostic message since this sample exceeds the unsigned long range in the 64-bit long or lower compiler systems no matter how evaluation is performed.</p>
<p>In C99, the #if type became the maximum integer type in the implementation.</p>
<p>Scoring: 2.</p>
<h4>i.35.3. Multi-character wide character constant evaluation</h4>
<p>There exists something called wide character multi-character constants. The evaluation method is also implementation-defined overall, however, a wide character multi-character constant needs to match the multi-character constant for corresponding multi-byte character.</p>
<p>Scoring: 0.</p>
<h3><a name="3.4.37" href="#toc.3.4.37">3.4.37. Translation limits</a></h3>
<p>Standard C specified the minimum for each translation limit that can be handled by compiler systems. However, this specification is quite lenient. Regarding 22 kinds of limitation values, each program including one or more examples meeting each limitation value must be processed and executed. As you can see in this Validation Suite sample, it is possible to write this program extremely simple and impractically with the minimum load for a compiler system. Please note that these translation limits are not guaranteed all the time. The translation limit specification is only an indication. These samples test only 8 kinds of translation limits regarding preprocessing. *1, *2, *3</p>
<p>A part of these test samples has lines wrapped to fit on a screen. These tests may fail in the compiler system that cannot process line splicing correctly (for example, Borland C.) Since line splicing testing is not the purpose here, please concatenate lines with an editor to re-test the sample if it fails.</p>
<p>Note:</p>
<pre>
*1 C90 5.2.4.1 Translation limits
*2 C99 5.2.4.1 Translation limits
</pre>
<p>In C99, translation limits are expanded to a large extent. They are even more so in the C++ Standard (refer to <a href=#4.6>4.6</a>.)</p>
<h4>n.37.1. 31 parameters in a macro definition</h4>
<p>In C90, up to 31 parameters in a macro definition are guaranteed in any case.</p>
<p>Scoring: 2.</p>
<h4>n.37.2. 31 arguments in a macro call</h4>
<p>Similarly, up to 31 arguments in a macro call are guaranteed in C90 in any case.</p>
<p>Scoring: 2.</p>
<h4>n.37.3. 31 characters in an identifier</h4>
<p>In C90, the top 31 characters of the internal identifier (including macro names) in a translation unit are guaranteed to be significant. Preprocessing goes without saying, but 31 byte names also need to be passed. *1</p>
<p>Scoring: 4.</p>
<p>Note:</p>
<pre>
*1 C90 6.1.2 Identifiers - Implementation limits
</pre>
<h4>n.37.4. 8 levels of nested conditional inclusion</h4>
<p>In C90, 8 levels of #if (#ifdef, #ifndef) section nesting is guaranteed in any case.</p>
<p>Scoring: 2.</p>
<h4>n.37.5. 8 levels of nested #include</h4>
<p>In C90, 8 levels of #include nesting is guaranteed in any case.</p>
<p>Scoring: 4.</p>
<h4>n.37.6. #if expression with 32 levels of parentheses</h4>
<p>In C90, 32 nesting levels of parentheses are guaranteed in an expression in any case. This seems to apply to the #if expressions as well (Different from generic expressions, it does not seem to be necessary to guarantee to that extent for the #if expressions. The only specifications which are defined as an exception are the #if expressions are integer constant expressions, which are evaluated only in long/unsigned long, do not require a query to execution environment, and that same evaluation methods as runtime or phase 7 are not required. Since the rest receive no special treatment, there are some parts where the specification seems somewhat extreme.)</p>
<p>Scoring: 2.</p>
<h4>n.37.7. 509 byte string literal</h4>
<p>In C90, the length of string literals is guaranteed up to 509 bytes. This length is that of a token and not the number of elements in a char array. It includes " in both ends and \n is counted as 2 bytes. In a wide string literal, the L prefix is included in the number.</p>
<p>Scoring: 2.</p>
<h4>n.37.8. 509 byte logical line</h4>
<p>In C90, the length of a logical line is guaranteed up to 509 bytes in any case.</p>
<p>Scoring: 2.</p>
<h4>n.37.9. 1024 macro definitions</h4>
<p>In C90, 1024 macro definitions are guaranteed in any case. This is the most ambiguous among specifications regarding translation limits. The amount of memory required by compiler systems for the simplest macros in this sample are totally different from that for many long macros. Also, test programs vary depending predefined macros are included in the 1024 macros. In an actual program, many macros are defined in standard headers before they are defined in a user program. This specification is truly only rough indication. The real limitations shall be determined by the memory amount, which can be provided by the system.</p>
<p>Scoring: 4.</p>
<h4>n.tlimit. C99 translation limits</h4>
<p>In C99, translation limits were extended to a large extent.</p>
<p>Scoring: 2 for each of below.</p>
<h4>n.37.1.L. 127 parameters in a macro definition</h4>
<h4>n.37.2.L. 127 arguments in a macro call</h4>
<h4>n.37.3.L. 63 characters in an identifier</h4>
<h4>n.37.4.L. 63 levels of nested conditional inclusion</h4>
<h4>n.37.5.L. 15 levels of nested #include</h4>
<h4>n.37.6.L. #if expression with 63 levels of parentheses</h4>
<h4>n.37.7.L. 4095 byte string literal</h4>
<h4>n.37.8.L. 4095 byte logical line</h4>
<h4>n.37.9.L. 4095 macro definitions</h4>
<br>
<h2><a name="3.5" href="#toc.3.5">3.5. Documentation of Implementation-Defined Items</a></h2>
<p>Standard C has items called implementation-defined. Specifications in these items vary depending on implementations. However, it is required that implementations describe specifications in a document. *1</p>
<p>Among the specifications that are implementation-defined, there are ones determined by CPU and OS in addition to the ones by compiler systems themselves. In case of cross compilers, CPU and OS may differ at translation and runtime.</p>
<p>The items below check if the implementation-defined items in preprocessing are described in a document. Since this is for preprocessing, it is for translation time as far as CPU and OS are concerned. d.1.* are for preprocessing specific specifications and d.2.* are related to compiler proper specifications. However, the #if expression evaluation can have different specifications from compiler proper.</p>
<p>Other than items below, there are some implementation-defined aspects in the #if expression evaluation. The first one is character sets (basic character set is ASCII or EBCDIC and such.) The integer representation (2's complement, 1's complement, or sign+absolute value) is another one. Same as a result converted from a signed integer into an unsigned by usual arithmetic conversion. However, since these are all machine and OS dependent, just those documents should be enough and they do not need to be described in the preprocessor documents. Therefore, these are not subject to scoring.</p>
<p>Note:</p>
<pre>
*1 C90 3 Definitions of Terms
C99 3 Terms, definitions, and symbols
</pre>
<h4>d.1.1. Header-name construction method</h4>
<p>A header-name is a special pp-token. How to combine sequences enclosed by &lt; and &gt; or " and " into one pp-token called a header-name is implementation-defined. It is easy since what is enclosed by " and " is treated as a pp-token called a string literal as far as implementation is concerned, however, what is enclosed by &lt; and &gt; has extremely special problems. That is because &lt;stdio.h&gt; is divided into 5 pp-tokens, &lt;, stdio, ., h, and &gt;, in translation phase 3 and composed into 1 pp-token in phase 4. In case this part is written using macros, even more delicate issues arise. *1</p>
<p>Scoring: 2. 2 points if this specification is described in implementation documents, 0 points otherwise.</p>
<p>Case-sensitivity in header-names and file name rules are also implementation-defined, however they do not necessarily need describing in implementation documents since these are OS dependent.</p>
<p>Note:</p>
<pre>
*1 C90 6.8.2 Source file inclusion -- Semantics
C99 6.10.2 Source file inclusion
</pre>
<h4>d.1.2. Header search method in #include</h4>
<p>How to search the header file after a header-name is taken out of the #include line is also implementation-defined. In case of the header-name enclosed by " and ", the files is searched in an implementation-defined method first and searched in the same way as the header-name enclosed by &lt; and &gt; if not found. However, the latter is also implementation-defined. This specification does not make any sense at all, however, it cannot be helped since Standard C cannot make assumptions regarding the OS.</p>
<p>In the OS with directory structures, the former searches the relative path to the current directory while the latter searches the directory specified by the implementation. However, some implementations search the relative path to the file that is the source of include. This cannot be said wrong, either, since it is implementation-defined. This is explained by the Rationale that the committee's intention is the specification of the search in the relative path to the current directory, but that it could not be officially defined since the OS cannot be assumed. *1</p>
<p>Also, the former search does not have to be supported (it is acceptable to treat the header-name enclosed by " and " in the same way as one enclosed by &lt; and &gt;.) The latter does not necessarily have to be a file, either. There may be a header built in implementations. *2</p>
<p>Scoring: 4. 4 points if these header search methods are fully described in documents, 2 points if not fully, and 0 point if almost no description.</p>
<p>Note:</p>
<pre>
*1 ANSI C Rationale 3.8.2 Source file inclusion
C99 Rationale 6.10.2 Source file inclusion
*2 C90 6.8.2 Source file inclusion -- Semantics
C99 6.10.2 Source file inclusion
</pre>
<h4>d.1.3. #include nesting limitation</h4>
<p>The number of the #include nesting level is implementation-defined. Though, at least 8 levels for C90 and 15 levels for C99 must be guaranteed. *1, *2</p>
<p>Scoring: 2.</p>
<p>Note:</p>
<pre>
*1 C90 5.2.4.1 Translation limits
C99 5.2.4.1 Translation limits
</pre>
<pre>
*2 C90 6.8.2 Source file inclusion -- Semantics
C99 6.10.2 Source file inclusion
</pre>
<h4>d.1.4. #pragma sub-directives that are implemented</h4>
<p>The #pragma preprocessing directive is a directive to specify enhanced functionalities specific to implementations. Enhanced functionalities must be all implemented as #pragma sub-directives in preprocessing as well. *1</p>
<p>Scoring: 4. 4 points if enough description is in documents regarding the #pragma sub-directives valid in the implementation (at least all #pragma sub-directives for preprocessing in preprocessing documents), 2 points if the descriptions are not enough, and 0 points if almost no description. Deduct 2 points if there is a directive specific to the implementation other than #pragma sub-directives (0 point is the lowest limit. The directives, which are prohibited by options specifying the closest specification to Standard C, are not included.)</p>
<p>Note:</p>
<pre>
*1 C90 6.8.6 Pragma directive
C99 6.10.6 Pragma directive
</pre>
<h4>d.pragma. Macro expansion in #pragma</h4>
<p>In C90, pp-tokens on the #pragma line was not subject to macro expansion. In C99, however, the line with the STDC token following #pragma is not subject to macro expansion and the macro expansion for the rest of #pragma sub-directives became implementation-defined.</p>
<p>Scoring: 2.</p>
<h4>d.1.5. Predefined macros</h4>
<p>Predefined macros other than <tt>__FILE__</tt>, <tt>__LINE__</tt>, <tt>__DATE__</tt>, <tt>__TIME__</tt>, <tt>__STDC__</tt>, <tt>__STDC_VERSION__</tt> (also <tt>__STDC_HOSTED__</tt> in C99) are implementation-defined. They must have a name with one _ followed by uppercase letters or starting with 2 _'s. *1</p>
<p>Scoring: 4. 2 points if the description is not enough. Deduct 2 points in case there is a predefined macro with the name which does not follow the specification restrictions (0 point is the lowest limit. The directives, which are prohibited by options specifying the closest specification to Standard C, are not included.)</p>
<p>Note:</p>
<pre>
*1 C90 6.8.8 Predefined macro names
C99 6.10.8 Predefined macro names
</pre>
<h4>d.predef. C99 predefined macros</h4>
<p>C99 added the macros, <tt>__STDC_IEC_559__</tt>, <tt>__STDC_IEC_559_COMPLEX__</tt>, and <tt>__STDC_ISO_10646__</tt>, which are predefined conditionally.</p>
<p><tt>__STDC_IEC_559__</tt> and <tt>__STDC_IEC_559_COMPLEX__</tt> are defined as 1 respectively in the implementation matching IEC 60559 floating point specification. These 2 are determined by the floating point operation library and it may be appropriate to define these in &lt;fenv.h&gt; or another actually. They do not necessarily have to be predefined in a preprocessor.</p>
<p><tt>__STDC_ISO_10646__</tt> is defined as a constant in the format such as 199712L which represents the year and month of the specification including amendments and corrigenda in complying ISO/IEC 10646 in case values of characters in the wchar_t type are all some sort of coded implementations of ISO/IEC 10646 (Universal Character Set of Unicode system.) This may also be defined in &lt;wchar.h&gt; or another and it does not seem to have to be predefined in a preprocessor.</p>
<p>In any case, the explanation is required in documents.</p>
<p>Scoring: 6. 2 points each for each of 3.</p>
<h4>d.1.6. Whether white-spaces should be compressed in phase 3</h4>
<p>In Standard C, tokenization is performed in translation phase 3. Whether a white-space sequence other than &lt;newline&gt; should be compressed into one space at that time is implementation-defined. *1</p>
<p>However, as this is an internal behavior that does not usually influence the compilation result, it is not a user concern. White-spaces in the beginning and end of a line may be removed.</p>
<p>If you are wondering this specification is unnecessary, it is not so, but there is one case necessary. That is in case the preprocessing directive line has [VT] and [FF]. This is defined as a specification in an incomprehensible way in Standard C. On one hand, this is treated as a violation of constraints; the specification above is established on the other hand. In other words, [VT] and [FF] can be compressed into one space together with spaces or tabs before and after in phase 3. In that case, [VT] and [FF] do not remain in phase 4, but they do remain in case they are not compressed to cause a violation of constraints.</p>
<p>I think it is enough if [VT] and [FF] handlings are described in documents.</p>
<p>Scoring: 2.</p>
<p>Note:</p>
<pre>
*1 C90 5.1.1.2 Translation phases 3
C90 6.8 Preprocessing directive -- Constraints
C99 5.1.1.2 Translation phases 3
C99 6.10 Preprocessing directive -- Constraints
</pre>
<h4>d.ucn. Whether UCN's \ should be repeated in stringizing</h4>
<p>In case stringizing pp-tokens including UCN by the # operator, whether UCN's \ should be repeated is implementation-defined. *1</p>
<p>Scoring: 2.</p>
<p>Note:</p>
<pre>
*1 C99 6.10.3.2 The # operator
</pre>
<h4>d.2.1. #if: Single-character character constant evaluation</h4>
<p>In general, the evaluation of character constant values is implementation-defined. This has several levels.</p>
<ol>
<li>What is the basic character set.<br>
<li>What are multi-byte character and wide character encodings.<br>
<li>How is sign handled.<br>
<li>What is the byte order for multi-byte character constant evaluation.<br>
</ol>
<p>Among these, 1 is not subject to scoring since they are OS dependent. The problems are 2, 3 and 4.</p>
<p>Even single byte single-character character constants are implementation-defined in terms of sign handling. Moreover, different evaluation between preprocessing and compilation is permitted. *1</p>
<p>Scoring: 2. 2 points if a document includes descriptions or if it has descriptions regarding the evaluation in the compilation phase and the same evaluation is performed in the #if expression. 0 points if no accurate description is written.</p>
<p>Note:</p>
<pre>
*1 C90 6.1.3.4 Character constants
C90 6.8.1 Conditional inclusion -- Semantics
C99 6.4.4. 4 Character constants
C99 6.10.1 Conditional inclusion -- Semantics
</pre>
<h4>d.2.2. #if: Multi-character character constant evaluation</h4>
<p>There is an issue of a byte order in multi-character character constant evaluation such as 'ab'. This is also implementation-defined.</p>
<p>Scoring: 2. The scoring method is same as d.2.1.</p>
<h4>d.2.3. #if: Multi-byte character constant evaluation</h4>
<p>Other than encoding, there are differences in sign handling and byte order in multi-byte character constant evaluation. These are all implementation-defined.</p>
<p>Scoring: 2. The scoring method is same as d.2.1.</p>
<h4>d.2.4. #if: wide character character constant evaluation</h4>
<p>Other than encoding, there are differences in sign handling and byte order in wide character constant evaluation. These are all implementation-defined.</p>
<p>Scoring: 2. The scoring method is same as d.2.1.</p>
<h4>d.2.5. #if: right shift for a negative number</h4>
<p>In general, how sign bits are handled in the right bit shift for a negative integer is implementation-defined. This depends on the CPU specification, but also the implementation of the compiler system. *1</p>
<p>Scoring: 2. The scoring method is same as d.2.1.</p>
<p>Note:</p>
<pre>
*1 C90 6.3.7 Bitwise shift operators -- Semantics
C99 6.5.7 Bitwise shift operators -- Semantics
</pre>
<h4>d.2.6. #if: Division and modulo of a negative number</h4>
<p>In general, in case one or both of right-hand side and left-hand sides are negative integers, the results of division and modulo are implementation-defined. This depends on the CPU specification, but also the implementation of the compiler system. *1, *2</p>
<p>Scoring: 2. The scoring method is same as d.2.1.</p>
<p>Note:</p>
<pre>
*1 C90 6.3.5 Multiplicative operators -- Semantics
C99 6.5.5 Multiplicative operators -- Semantics
</pre>
<p>*2 In C99, quotients are rounded off to the direction of 0 just as div() and ldiv().</p>
<h4>d.2.7. Valid length of identifiers</h4>
<p>Up to which byte of the identifier including a macro name from the beginning is significant is implementation-defined. For macro names and internal identifiers, 31 bytes for C90 and 63 bytes for C99 must be guaranteed. *1</p>
<p>Scoring: 2.</p>
<p>Note:</p>
<pre>
*1 C90 6.1.2 Identifiers -- Implementation limits
C99 6.4.2.1 Identifiers -- General -- Implementation limits
</pre>
<h4>d.mbident. Multi-byte characters in identifiers</h4>
<p>In C99, implementations using multi-byte characters in identifiers are permitted. This specification is implementation-defined. *1</p>
<p>Scoring: 2.</p>
<p>Note:</p>
<pre>
*1 C99 6.4.2.1 Identifiers -- General
</pre>
<br>
<h1><a name="4" href="#toc.4">4. Evaluation of Aspects Unspecified by Standard</a></h1>
<p>Even if something is not required to implementations by Standards, they may be important in order to evaluate the "quality" of implementations. In this chapter, quality evaluation testing is explained.</p>
<br>
<h2><a name="4.1" href="#toc.4.1">4.1. Multi-byte Character Encoding</a></h2>
<p>There are various encodings for multi-byte character and wide character. The specification of encoding is implementation-defined. In this section, however, the "quality" matter of implementations are explained, that is how many encodings the implementation can handle and to what degrees can handle.</p>
<p>This Validation Suite provides samples supporting several encodings for m_33, m_34, and m_36. Compiler systems not only have to support the standard encoding on the system, but also must support many encodings in order to process source files supporting multiple languages. *1</p>
<p>However, the method of testing differs depending on the system and compiler systems and is not easy.</p>
<p>In GCC, use environment variables, LC_ALL, LC_CTYPE, and LANG, change behavior, but their implementations are half-finished and not reliable. Moreover, whether this feature is available depends on the configuration when GCC itself was compiled.</p>
<p>GCC 3.4 changed its way of multi-byte character processing entirely. It converts every encoding of source file to UTF-8 at the start of preprocessing (so to say the translation phase 1). Hence, a multi-byte character constant in #if expression cannot be evaluated in other than UTF-8 which is irrelevant to the original encoding.</p>
<p>The C++98 Standard has a similar problem. Since it stipulates to convert multi-byte character into UCN at translation phase 1, multi-byte character constant in #if expression cannot be evaluated in other than UCN.</p>
<p>Considering the rather confused status of the Standards and the implementations, and considering the lack of portability and lack of meaning of character constant in #if expression, I excluded the tests of multi-byte/wide character constants in #if expression (m.33, m.34, m.36.1) from the subject of scoring since Validation Suite V.1.5.</p>
<p>In Visual C++, use #pragma setlocale, which is a convenient directive. On Windows, "Regional and Language Options" is supposed to change the language to use but is ill-defined and cumbersome. #pragma setlocale is convenient for a programmer since it can be used without tampering with Windows (though how well Visual C++ itself implements it is a separate story).</p>
<p>As far as other compiler systems I have tested are concerned, they support only their default encoding only. Many of them support functions such as setlocale() in the library, which have nothing to do with preprocessing or compiling source files. What is necessary is a capability for a compiler system itself to recognize and process the encoding of source files.</p>
<p>Note:</p>
<p>*1 In C99, Unicode sequences starting with \u or \U were introduced which made it difficult to understand the relationship with multi-byte/wide character. The C++ Standard is even more complicated.</p>
<h4>m.33.1. Wide character constant evaluation</h4>
<p>For wide character constant, see <a href="#3.4.33">3.4.33</a>.</p>
<p>Scoring: none.</p>
<h4>m.34. Multi-byte character constant evaluation</h4>
<p>Multiple byte multi-byte character constants are supported in the #if expressions (in Standards, the term, multi-byte characters, include single-byte characters, however, in this document, they refer to only multi-byte characters which are not single-byte to avoid confusion.) The evaluation is even more implementation-defined than single-byte character constants.</p>
<p>Scoring: none.<br>
This test needs to be determined along with the test in <a href="#u.1.7">u.1.7</a> described later. Simply evaluating a character value does not mean recognizing an encoding. u.1.7 tests whether a multi-byte character is in the range accepted for the encoding. The encoding of a character is properly recognized only after its value is correctly evaluated in m.34 and after an appropriate diagnostic message is displayed in u.1.7.</p>
<h4>m.36.1. 0x5c in Multi-byte Characters is not an escape character</h4>
<p>If the encoding of (multi-byte | wide) characters is shift-JIS, Big-5, or ISO-2022-*, cumbersome issues arise as there may be a byte with the 0x5c value in a character which is same as '\\'. Compiler systems must not interpret this as a \ (backslash) escape character. A (multi-byte | wide) character is one character and not two single-byte characters.</p>
<p>0x5c in the value for multi-byte character constants must not be interpreted as the beginning of an escape sequence.</p>
<p>Scoring: none.</p>
<h4>m.36.2. The # operator does not insert \ in Multi-byte Characters</h4>
<p>'\\' must not be inserted where a multi-byte character with 0x5c is included in an argument of an operand for the # operator. Though there is a method of supporting multi-byte characters, not supported in the compiler proper, in a preprocessor by inserting '\\', is at another level.</p>
<p>In addition, there are troublesome issues different from other literals in the tokenization for this type of character constant and string literal including multi-byte characters.</p>
<p>As multi-byte characters encoded in ISO-2022-* include not only '\\', but also bytes of values matching '\'' or '"', sloppy processing will end up with a tokenization failure.</p>
<p>Scoring: 7. 2 points each for the Shift-JIS and Big-5 encoding. 1 point each for 3 samples of ISO-2022-JP.</p>
<p>This item needs to test not only m_36_*.t, but also m_36_*.c. This is because the preprocessor which correctly processes stringizing may fail identifying the complex string literal including a Kanji character with 0x5c as an argument in the assert() macro. m_36.c also tests the tokenization of string literals.</p>
<p>GCC 3.4-4.1 converts the encodings to UTF-8, and often fails to convert back to the original encoding. Though the GCC's conversions are usually unwanted ones, these testcases do not check such conversions, they check only the handling of 0x5c byte (lenient tests.)</p>
<br>
<h2><a name="4.2" href="#toc.4.2">4.2. Undefined Behavior</a></h2>
<p>In Standard C, there are many specifications of an undefined behavior. What causes an undefined behavior is incorrect programs or data or at least programs without portability. However, it is not mandatory to issue a diagnostic message, different from violation of syntax rules or constraints. Implementations can process this in any way. Some sort of reasonable processing as a valid program or handling it as an error by issuing a diagnostic message are acceptable, and it is not against Standards even if the process is canceled or run out of control without issuing a diagnostic message.</p>
<p>However, in order to evaluate the "quality" of an implementation, concretely what undefined behavior is a question. I think it is appropriate for an implementation to issue some sort of diagnostic message. Not to mention an error case, even in the case of handling it as a valid program, it is helpful to issue a warning in order to indicate portability issues. Runaway is out of the question.</p>
<p>The following tests check whether implementations issue an appropriate diagnostic message for source that causes an undefined behavior. Diagnostic messages may be an error or warning. In case of a warning, some sort of reasonable processing is necessary.</p>
<p>u.1.* are preprocessing specific problems and u.2.* are common in constant expressions in general.</p>
<p>Scoring: 1 point each for an item unless otherwise noted if an appropriate diagnostic message is issued. 0 points for an off-center or no diagnostic message.</p>
<h4>u.1.1. Source files not ending with &lt;newline&gt;</h4>
<p>Source files which are not empty and whose end is not the &lt;newline&gt; cause undefined behavior (Depending on OS, no newline character data exists in a file and a newline is automatically added by the implementation when a file is read.) *1</p>
<p>u.1.1, u.1.2, u.1.3, and u.1.4 are all ones with incomplete source files. In case the translation unit ends within the file, most implementations seem to issue a diagnostic message. However, even with such implementations, there is a possibility that the incomplete file is treated as valid source by processing the file included within another. Although this is also a type of undefined behavior and not an erroneous process, it is appropriate to issue a diagnostic message.</p>
<p>Note:</p>
<pre>
*1 C90 5.1.1.2 Translation phases
C99 5.1.1.2 Translation phases
</pre>
<h4>u.1.2. Source files ending with &lt;backslash&gt;&lt;newline&gt;</h4>
<p>Source files ending with the &lt;backslash&gt;&lt;newline&gt; sequence cause an undefined behavior. *1</p>
<p>Note:</p>
<pre>
*1 C90 5.1.1.2 Translation phases
C99 5.1.1.2 Translation phases
</pre>
<h4>u.1.3. Source files ending in the middle of comments</h4>
<p>Source files ending in the middle of comments cause an undefined behavior. This is actually the case of unclosed or nested comments. *1</p>
<p>Note:</p>
<pre>
*1 C90 5.1.1.2 Translation phases
C99 5.1.1.2 Translation phases
</pre>
<h4>u.1.4. Source files ending in the middle of a macro call</h4>
<p>Source files ending with an incomplete macro call is undefined. *1</p>
<p>This occurs in case that a parenthesis is missing to close a macro argument and it is important to have a diagnostic message.</p>
<p>Note:</p>
<pre>
*1 C90 6.8.3.4 Rescanning and further replacement
C99 6.10.3.4 Rescanning and further replacement
</pre>
<h4>u.1.5. Invalid character in places other than quotes</h4>
<p>Extremely limited characters can be written in places other than string literals, character constants, header-names, and comments. They are uppercase and lowercase alphabets, numbers, 29 symbols, and 6 kinds of white space characters. It is no wonder since these are for source. *1</p>
<p>Here, we test the case of control codes other than white spaces. Although control codes are invalid even in string literals and elsewhere, compiler proper can check this. As there are many locale-specific or implementation-defined aspects in character sets in general and the range is not necessarily clear, we do not test those. Kanji characters are undefined in places other than above, but not tested for similar reasons.</p>
<p>Note:</p>
<pre>
*1 C90 5.2.1 Character sets
C99 5.2.1 Character sets
</pre>
UCN was added in C99.</p>
</blockquote>
<h4><a name="u.1.6">u.1.6. [VT][FF] on control lines</a></h4>
<p>Preprocessing directive lines starting with #, even if white space characters, cause a violation of constraints when other than [SPACE][TAB] exists. However, this is the case of translation phase 4 and it is possible to compress those with a sequence of white spaces before and after which are other than &lt;newline&gt; into one space in phase 3 prior to phase 4. In such event, no violation occurs. *1</p>
<p>So as in Standards, it is appropriate to issue a diagnostic message to this. This is not an undefined behavior test, however, it is included here as a matter of convenience since it is difficult to classify this elsewhere.</p>
<p>Note:</p>
<pre>
*1 C90 5.1.1.2 Translation phases
C99 5.1.1.2 Translation phases
C90 6.8 Preprocessing directives -- Constraints
C99 6.10 Preprocessing directives -- Constraints
</pre>
<h4><a name="u.1.7">u.1.7. Invalid multi-byte character sequences in quotes</a></h4>
<p>Even inside a string literal, character constant, header-name, or comment, a sequence that is not accepted as multi-byte characters causes undefined behavior. This is the case where the next byte of the first byte of a multi-byte character cannot be used as the second byte. *1</p>
<p>Scoring: 9. 1 point each for 6 types of encodings other than UTF-8, 3 points for UTF-8.
The UTF-8 testcase has 4 testing lines: the first is normal sequence, and the rest 3 are illegal ones.
Give 1 point each for the 3 illegal lines diagnosed appropriately.
Give no point to a preprocessor which issues off-target message at the normal case.
Also, refer to the explanation of m.34.</p>
<p>Note:</p>
<pre>
*1 C90 5.2.1.2 Multibyte characters
C99 5.2.1.2 Multibyte characters
</pre>
<h4>u.1.8. Logical lines ending in the middle of a character constant</h4>
<p>The character constant pp-token must complete on that logical line. If there is a ' without corresponding ' on a logical line, it is undefined. *1</p>
<p>An optional message can be written on the #error line. However, it must be arranged in the pp-token sequence and a single apostrophe is not allowed. In this sample, the part intended for a comment is eaten by searching the ' which should be at the end of the character constant.</p>
<p>Note:</p>
<pre>
*1 C90 6.1 Lexical elements -- Semantics
C99 6.4 Lexical elements -- Semantics
</pre>
<h4>u.1.9. Logical lines ending in the middle of a string literal</h4>
<p>String literals must complete on that logical line. A single " is undefined. *1</p>
<p>String literals extending over lines seem to have been accepted in many UNIX compiler systems. Source files expecting it can still be seen sometimes.</p>
<p>Note:</p>
<pre>
*1 C90 6.1 Lexical elements -- Semantics
C99 6.4 Lexical elements -- Semantics
</pre>
<h4>u.1.10. Logical lines ending in the middle of a header name</h4>
<p>Incomplete header-names on the #include logical line is also undefined. *1</p>
<p>Note:</p>
<pre>
*1 C90 6.8.2 Source file inclusion -- Semantics
C99 6.10.2 Source file inclusion -- Semantics
</pre>
<h4>u.1.11. ', ", /*, or \ in a header name</h4>
<p>', /*, or \ in the header-name pp-token is undefined. The case " exists in the header-name enclosed by &lt; and &gt; is also the same (error since " becomes the end of the pp-token from the beginning in the header-name of a string literal format.) *1</p>
<p>These, except for \, are all confusable with a character constants, string literal, or comment and can be interpreted either way.</p>
<p>\ is mistakable with an escape sequence. Though a header-name has no escape sequence, this is not determined as a header-name until phase 4 after tokenization in translation phase 3. Therefore, implementations suffer from identifying this case. Though escape sequences are processed in phase 6, it is necessary to recognize an escape sequence since \" is interpreted as an escape sequence rather than the end of a string literal in phase 3.</p>
<p>However, \ is a Standard path-delimiter in the OS such as DOS/Windows and implementations on those OS certainly handle this as a valid character (except for the case that the last character is \ in the header-name of a string literal format.) Many cases causing an undefined behavior are erroneous programs, but not always so. It is not recommended to write \ where / is adequate. Implementations should issue a warning. On other OS's, it will be an error when opening files; no diagnosis is necessary in preprocessing tokenization.</p>
<p>Note:</p>
<pre>
*1 C90 6.1.7 Header names -- Semantics
C99 6.4.7 Header names -- Semantics
</pre>
<h4>u.1.12. #include argument is not a header name</h4>
<p>It is undefined if the argument on the #include line is not a header-name. In other words, this is the case where an argument is not in the string literal format or not enclosed by &lt; and &gt;, or no macro which is expanded into either of these exists. *1</p>
<p>Note:</p>
<pre>
*1 C90 6.8.2 Source file inclusion -- Semantics
C99 6.10.2 Source file inclusion -- Semantics
</pre>
<h4>u.1.13. Extra token in the #include argument</h4>
<p>The argument on the #include line is one header-name only. Extra pp-token other than that is undefined. *1</p>
<p>Note:</p>
<pre>
*1 C90 6.8.2 Source file inclusion -- Semantics
C99 6.10.2 Source file inclusion -- Semantics
</pre>
<h4>u.1.14. Missing line number in the #line argument</h4>
<p>It is undefined if the argument on the #line has no line number (A file name is optional. A line number must be the first argument.) *1</p>
<p>Note:</p>
<p>*1 The sources of Standards up to u.1.18 are all same below.</p>
<pre>
C90 6.8.4 Line control -- Semantics
C99 6.10.4 Line control -- Semantics
</pre>
<h4>u.1.15. The #line file name argument is not a string literal</h4>
<p>The file name which is the second argument of #line must be a string literal.</p>
<p>If this is a wide string literal, it is a violation of constraints while the rest of the #line errors are undefined. This specification lacks balance.</p>
<h4>u.1.16. Extra token in the #line arguments</h4>
<p>Three or more arguments on the #line line cause undefined errors.</p>
<h4>u.1.17. The line number argument for #line is out of [1, 32767] range</h4>
<p>In C90, the line number argument for #line must be in the range of [1, 32767]. It is undefined otherwise.</p>
<p>This sample tests the case where the #line specification is within this range, but the line number of the source exceeds the range afterward. Depending on implementations, the line number may silently wrap. It is appropriate to issue a warning.</p>
<p>Scoring: 2. 1 point if 1 or 2 out of 3 samples can be diagnosed.</p>
<h4>u.line. C99: the line number argument for #line is out of range</h4>
<p>In C99, this range is [1, 2147483647].</p>
<h4>u.1.18. The line number argument for #line is not decimal</h4>
<p>The line number argument for #line must be a decimal number. Hexadecimal and others are undefined.</p>
<h4><a name="u.1.19">u.1.19. Macro on the #if line expanded into defined</a></h4>
<p>The fact that 'defined' is an operator and confusable with an identifier causes various problems. Since it is once tokenized as an identifier in translation phase 3 and recognized as an operator in phase 4 only if this exists on the #if line, defining a macro which is expanded into defined on the #define line is possible. If this macro actually appears on the #if line, it is undefined. There is no guarantee for the result of its expansion to be treated as an operator. *1</p>
<p>Defining a macro named defined is itself undefined (refer to <a href="#u.1.21">u.1.21)</a>, however, such an example is not seen in reality. Although, the macro definition which has a token named defined in the replacement list can be seen from time to time. Some compiler systems treat this as legitimate by performing a special process, which is not a logical specification.</p>
<p>Scoring: 2. 1 point if only 1 out of 2 samples can be diagnosed.</p>
<p>Note:</p>
<pre>
*1 C90 6.8.1 Conditional inclusion -- Semantics
C99 6.10.1 Conditional inclusion -- Semantics
</pre>
<h4>u.1.20. defined, __LINE__, etc. for the #undef argument</h4>
<p>It is undefined if the #undef argument is defined, <tt>__LINE__</tt>, <tt>__FILE__</tt>, <tt>__DATE__</tt>, <tt>__TIME__</tt>, <tt>__STDC__</tt>, or <tt>__STDC_VERSION__</tt>. *1, *2, *3</p>
<p>Note:</p>
<pre>
*1 C90 6.8.8 Predefined macro names
C99 6.10.8 Predefined macro names
</pre>
<pre>
*2 Amendment 1/3.3 Version macro
</pre>
<p>*3 <tt>__STDC_HOSTED__</tt>, <tt>__STDC_ISO_10646__</tt>, <tt>__STDC_IEC_559__</tt>, and <tt>__STDC_IEC_559_COMPLEX</tt> were added in C99.</p>
<h4><a name="u.1.21">u.1.21. defined, __LINE__, etc. for the #define macro name</a></h4>
<p>It is undefined if the macro name defined by #define is defined, <tt>__LINE__</tt>, <tt>__FILE__</tt>, <tt>__DATE__</tt>, <tt>__TIME__</tt>, <tt>__STDC__</tt>, or <tt>__STDC_VERSION__</tt>. *1, *2, *3</p>
<p>Scoring: 2. 1 point if 2 out of 3 samples are diagnosed.</p>
<p>Note:</p>
<pre>
*1 C90 6.8.8 Predefined macro names
</pre>
<pre>
*2 Amendment 1/3.3 Version macro
</pre>
<p>*3 <tt>__STDC_HOSTED__</tt>, <tt>__STDC_ISO_10646__</tt>, <tt>__STDC_IEC_559__</tt>, and <tt>__STDC_IEC_559_COMPLEX</tt> were added in C99.</p>
<h4>u.1.22. Invalid pp-token generation by the ## operator</h4>
<p>The result of pp-token concatenation by the ## operator must be a valid (single) pp-token. It is undefined, otherwise. *1</p>
<p>In this sample, the subject matter is the pp-token called a pp-number which was defined as a new specification in Standard C.</p>
<p>Note:</p>
<pre>
*1 C90 6.8.3.3 The ## operator -- Semantics
C99 6.10.3.3 The ## operator -- Semantics
</pre>
<h4>u.concat. C99: Invalid pp-token generation by the ## operator</h4>
<p>The // mark is used for a comment in C99 and C++, but is not a pp-token. The result of generating this sequence with the ## operator is undefined.<br>
A comment is converted into a space before a macro is defined or prior to expansion, thus it is not possible for a macro to generate a comment.</p>
<h4>u.1.23. Invalid pp-token generation by the # operator</h4>
<p>The result of pp-token concatenation by the # operator must be a valid (single) string literal. It is undefined otherwise. *1</p>
<p>This problem rarely occurs. As you can see in this sample, it is limited to odd cases where \ is outside of a literal and even more special cases. This sample shall be diagnosed in the compilation phase rather than preprocessing and that is enough. However, implementations should not crash or silently ignore the problem.</p>
<p>Note:</p>
<pre>
*1 C90 6.8.3.2 The # operator -- Semantics
C99 6.10.3.2 The # operator -- Semantics
</pre>
<h4>u.1.24. Empty argument in a macro call</h4>
<p>Having an empty argument in a function-like macro call is undefined in C90. *1</p>
<p>Performing reasonable macro expansion by interpreting an empty argument as 0 pieces of token is very possible. *2</p>
<p>This is an example where implementations can give the undefined specification a meaningful definition. Such implementations, however, also have no portability in pre-C99 at least. It will be appropriate to issue a warning.</p>
<p>Scoring: 2. 1 point if 3 of 4 out of 5 samples are diagnosed.</p>
<p>Note:</p>
<pre>
*1 C90 6.8.3 Macro replacement -- Semantics
*2 C99 6.10.3 Macro replacement -- Semantics
</pre>
<h4><a name="u.1.25">u.1.25. Argument similar to the control line in a macro call</a></h4>
<p>A function-like macro call can extend over multiple logical lines. Therefore, there may be a line confusable with a preprocessing directive in an argument; the result is undefined. *1</p>
<p>This type of "argument" will be interpreted as a preprocessing directive if it is in the #if group to be skipped.</p>
<p>Note:</p>
<pre>
*1 C90 6.8.3 Macro replacement -- Semantics
C99 6.10.3 Macro replacement -- Semantics
</pre>
<h4><a name="u.1.26">u.1.26. Macro expansion ending with a function-like macro name</a></h4>
<p>In C90, the macro expansion result ending with a function-like macro name is considered to be undefined. This is an interpretation added later, but not clear in meaning. Please refer to <a href="#2.7.6">2.7.6</a>. *1</p>
<p>This was included as a test in this Validation Suite up to V.1.2, but omitted from V.1.3.</p>
<p>Note:</p>
<p>*1 ISO/IEC 9899:1990 / Corrigendum 1</p>
<p>This specification was removed in C99 and a complicated specification was added.</p>
<h4><a name="u.1.27">u.1.27. Invalid directives</a></h4>
<p>In case the first pp-token on a line is # and there is another pp-token after that, what follows # must be a preprocessing directive usually. The line of # only is accepted for some reason. *1</p>
<p>However, # at the beginning of a line or followed by an invalid directive or other pp-token are not violations of syntax or constraint for preprocessing. As you can see in <a href=#u.1.25>u.1.25</a>, that is because all lines starting with # do not have to be preprocessing directive lines. Which one is a preprocessing directive line depends on the context.</p>
<p>Standards do not specify this as undefined, however, it is in a way undefined in the sense that "it is not defined as a specification." It is desirable for an implementation to issue some sort of diagnostic message. However, this does not necessarily have to be diagnosed by preprocessing. If preprocessing outputs this line as is, it will be an error in the compilation phase. That is acceptable as well. There is no danger as long as preprocessing avoids the case where it interprets silently #ifdefined as #if defined.</p>
<p>In C99, something not clear in the meaning called # non-directive was added. However, its content is not defined and I can say it is undefined, practically speaking. *2</p>
<p>Note:</p>
<pre>
*1 C90 6.8 Preprocessing directives -- Syntax
*2 C99 6.10 Preprocessing directives -- Syntax
</pre>
<h4>u.1.28. Directive name not macro-expanded</h4>
<p>In order for the line starting with # to be a preprocessing directive, only a directive name is allowed as the next pp-token. Directive names are never macro-expanded.</p>
<p>In the case where an identifier which is not a directive name comes after # and it is a macro name, there are 2 processing steps; diagnosing it as a missing directive and considering it as usual text to expand a macro for an output. Even the latter needs some sort of diagnosis. Since the latter should cause an error in the compilation phase, that is acceptable as well. Only processing what is macro-expanded as a "correct" preprocessing directive should not happen.</p>
<h4>u.2.1. Undefined character escape sequence in the #if expression</h4>
<p>Character escape sequences in a string literal or character constant support only \', \", \?, \\, \a, \b, \f, \n, \r, \t, and \v as a specification. Other character sequences starting with \ are undefined. Especially, sequences with \ followed by a lowercase are reserved for future use. *1</p>
<p>Many of these diagnoses are handled by the compilation phase, however, only preprocessing diagnoses the case where these exist in a character constant for the #if expression.</p>
<p>Note:</p>
<pre>
*1 C90 6.1.3.4 Character constants -- Description
C99 6.4.4.4 Character constants -- Description
C90 6.9.2 Character escape sequences
C99 6.11.4 Character escape sequences
</pre>
<p>In C99, UCN (universal-character-name) escape sequences in the \uxxxx or \Uxxxxxxxx format were added.</p>
<h4>u.2.2. Bit shift operation with invalid shift count in the #if expression</h4>
<p>In a bit shift operation for a integer type, the case that the right operand (shift count) is a negative value or greater than or equal to the width of the left operand type is undefined. *1</p>
<p>If this is in the #if expression, it should be diagnosed by preprocessing.</p>
<p>Note:</p>
<pre>
*1 C90 6.3.7 Bitwise shift operators -- Semantics
C99 6.5.7 Bitwise shift operators -- Semantics
</pre>
<br>
<h2><a name="4.3" href="#toc.4.3">4.3. Unspecified Behavior</a></h2>
<p>In Standard C, there is a specification called unspecified. This is for valid programs without a processing method unspecified and whose processing method is not necessarily documented by the implementation.</p>
<p>There are not many examples for this. Only extremely special cases have different results according to the processing method. However, it is desirable to issue a warning even if it is a special case.</p>
<p>The result of a program depending on unspecified behavior is undefined.</p>
<p>It is unspecified in preprocessing and the ones which have different results according to processing methods are the following. In these 2 tests, 2 points are given for each, whether an invalid pp-token is generated and a diagnostic message is issued or whether a warning for no portability is issued. In case of the latter, it can be at macro definition or expansion.</p>
<p>Additionally, evaluation order within an #if expression is unspecified. This is not included in testing since the #if expression result does not change because of this.</p>
<h4>s.1.1. Unspecified evaluation order of the # and ## operators</h4>
<p>In case both the # and ## operators are in one macro definition, which is evaluated first is not specified. *1</p>
<p>This sample is the example of different results depending on which of # and ## is evaluated first. Furthermore, if ## is evaluated first, # is treated as a pp-token rather than an operator, causing the concatenation result to generate an invalid pp-token. As this type of macro does not have portability, it is preferable for an implementation to issue a warning.</p>
<p>Scoring: 2.</p>
<p>Note:</p>
<pre>
*1 C90 6.8.3.2 The # operator -- Semantics
C99 6.10.3.2 The # operator -- Semantics
</pre>
<h4>s.1.2. Unspecified evaluation order for multiple ## operators</h4>
<p>In case there are multiple ## operators in one macro definition, the evaluation order is not specified. *1</p>
<p>In this sample, an invalid pp-token is generated on the way of evaluation depending on the evaluation order. As this type of macro does not have portability, it is preferable for an implementation to issue a warning.</p>
<p>Scoring: 2.</p>
<p>Note:</p>
<pre>
*1 C90 6.8.3.3 The ## operator -- Semantics
C99 6.10.3.3 The ## operator -- Semantics
</pre>
<br>
<h2><a name="4.4" href="#toc.4.4">4.4. Other Cases Where a Warning is Preferable</a></h2>
<p>Other than undefined and unspecified, there are cases for which it is preferable for implementations to issue a warning. Those are below.</p>
<p>The w.1.* and w.2.* are totally valid programs as far as Standards are concerned, however, they have probably some sort of errors and diagnosis is important. w.1.* are preprocessing specific problems and w.2.* are #if expression versions of the problems common with operations in the compilation phase.</p>
<p>The w.3.* concern a specification with implementation-defined aspects of translation limits. Implementing the translation limits beyond the minimum value guaranteed by a Standard can be said to improve the quality of the implementation. On the other hand, programs depending on it will have the problem of limited portability. Therefore, it is preferable for an implementation which implements translation limits beyond the minimum value to issue warnings.</p>
<p>In tests below, it is a pass if an appropriate diagnostic message is issued. w.3.* allow an error when the translation limits of an implementation match the minimum value of the Standard. It is a failure if an error occurs without meeting the minimum value (whether meeting the minimum value is determined in n.37.*.)</p>
<h4>w.1.1. /* inside a comment</h4>
<p>There are many nested comments and typos in source missing a comment mark. Among those, nested comments, /* /* */ */, and a comment with only */ always cause an error in the compilation phase since the */ sequence does not exist in the C language. However, when */ is missing, it may not cause an error since the part up to the end of the next comment is interpreted as a comment. This is a dangerous mistake and it is important for preprocessing to issue a warning. Even if the case some sort of error occurs in the compilation phase, the cause is no longer clear at that time.</p>
<p>Scoring: 4.</p>
<h4>w.1.2. Inclusion of a subsequent token sequence by macro rescan</h4>
<p>The case of the inclusion of a token sequence after a replacement list at macro rescan is an unwritten specification in K&amp;R 1st and the official Standard C. However, this situation is brought about only by an unusual macro. Especially, the one where the replacement list comprises the first half part of another function-like macro call is an extremely unusual macro. In reality, the possibility of an error in a macro definition is large, it is preferable for the implementations to issue a warning. I sometimes see an object-like macro expanded into a function-like macro name though such writing style causes bad readability.</p>
<p>This is a problem of Standard. I think that incomplete rescan in a replacement list should be specified as an error (violation of constraints) (refer to <a href="#2.7.6">2.7.6</a>.)</p>
<p>Scoring: 4. 1 point if only 1 out of 2 samples can be diagnosed.</p>
<h4>w.2.1. Negative value converted into an unsigned number by the usual arithmetic conversion in the #if expression</h4>
<p>In the mixed mode operation of signed and unsigned integers, "usual arithmetic conversion" is performed and signed numbers are converted into unsigned numbers. In case the original signed integer is positive, the value does not change. However, if it is negative, it is converted into a large positive number. This is not an error, but not normal and there is a possibility of an error. It is preferable to issue a warning. In preprocessing, this phenomenon is seen in the #if expression.</p>
<p>Scoring: 2.</p>
<h4>w.2.2. Unsigned operation in the #if expression wraps round</h4>
<p>In the case the result of an unsigned operation is out of the range, it may not be an error since it is supposed to be wrapped round. Since there is a possibility of an error, it is preferable to issue a warning.</p>
<p>Scoring: 1.</p>
<h4>w.3.1. Identifier of 32 bytes or longer, 32 or more macro parameters, 32 or more macro arguments</h4>
<p>w.3.1, w.3.2, w.3.3, w.3.4, and w.3.5 are all tests regarding the translation limits in C90. The contents are clear by themselves and no explanation is necessary. Compare these with n.37.*.</p>
<p>Scoring: 3. 1 point each for each of 3 items.</p>
<h4>w.3.2. 9 or more levels of #if (#ifdef, #ifndef) nesting</h4>
<p>Scoring: 1.</p>
<h4>w.3.3. 9 or more levels of #include nesting</h4>
<p>Scoring: 1.</p>
<h4>w.3.4. 33 or more sub-expression nesting in the #if expressions</h4>
<p>Scoring: 1.</p>
<h4>w.3.5. 510 bytes or longer string literals</h4>
<p>Scoring: 1.</p>
<h4>w.3.6. 510 bytes or longer logical lines</h4>
<p>Scoring: 1.</p>
<h4>w.3.7. 1025 or more macro definitions</h4>
<p>Only this is same as <a href="#n.37.8">n.37.8</a>. This is the most approximate one in the specification for translation limits. Whether built-in macros should be counted in 1024 macros and whether the macros defined in standard headers should be counted vary the number. The macro tested in this sample is the 1024th in header files, but there are some macros defined prior in warns.t and warns.c. In any case, this macro exceeds the 1024th. Implementations are expected to issue warnings at appropriate places.</p>
<p>Scoring: 1.</p>
<h4>w.tlimit. C99 translation limits</h4>
<p>In C99, translation limits were largely extended. It is preferable even for the implementation with the specification exceeding this to issue a warning for the source exceeding the specification for the portability.</p>
<p>Scoring: 9. 1 point each for the following 9 items.</p>
<p>Test samples are in the test-l directory. l_37_8.c is pseudo source which can be preprocessed, but not compiled.</p>
<h4>w.3.1.L. 128 or more macro parameters</h4>
<h4>w.3.2.L. 128 or more macro arguments</h4>
<h4>w.3.3.L. 64 bytes or longer identifier</h4>
<h4>w.3.4.L. 64 or more levels of #if (#ifdef, #ifndef) nesting</h4>
<h4>w.3.5.L. 16 or more levels of #include nesting</h4>
<h4>w.3.6.L. 64 or more levels of sub-expression nesting in the #if expressions</h4>
<h4>w.3.7.L. 4096 bytes or longer string literals</h4>
<h4>w.3.8.L. 4096 bytes or longer logical lines</h4>
<h4>w.3.9.L. 4096 or more macro definitions</h4>
<br>
<h2><a name="4.5" href="#toc.4.5">4.5. Other Quality Matters</a></h2>
<p>Below is the items concerning the quality such as implementation's ease of use. Cases other than q.1.1 cannot be tested by sample programs.</p>
<p>q.1.* are regarding behaviors.<br>
q.2.* are regarding options and extended functionalities.<br>
q.3.* are regarding the runnable systems and the efficiency on them.<br>
q.4.* are regarding documents.</p>
<p>Among these, some cannot help relying on rather subjective evaluation. Others can be evaluated objectively, but the measure may not be specified clearly. Refer to <a href="#6.2">6.2</a> to score them appropriately.</p>
<h3><a name="4.5.1" href="#toc.4.5.1">4.5.1. Qualities regarding Behaviors</a></h3>
<h4>q.1.1. Translation limits above the specification</h4>
<p>Concerning translation limits, the minimum specification is defined leniently in Standards. However, the actual implementations should have the specification above this. Especially, the requirement of nesting levels of #if and #include are too low in C90.</p>
<p>In C99, translation limits were largely raised. Also, restricting identifier length to less than 255 bytes is described as an obsolescent feature.</p>
<p>In the q.* items, only these are prepared as test samples. l_37_?.t and l_37_?.c in the test-l directory each test the translation limits as below. These exceed the C99 specification (however, the translation limit values in the C++ Standard guideline are higher than these.)</p>
<blockquote>
<table>
<tr><th>37.1L</th><td>Number of parameters in a macro definition </td><td>255</td></tr>
<tr><th>37.2L</th><td>Number of arguments in a macro call </td><td>255</td></tr>
<tr><th>37.3L</th><td>Length of an identifier </td><td>255 bytes</td></tr>
<tr><th>37.4L</th><td>Nesting level for #if </td><td>255</td></tr>
<tr><th>37.5L</th><td>Nesting level for #include </td><td>127</td></tr>
<tr><th>37.6L</th><td>Nesting level for sub-expressions in the #if expressions</td><td>255</td></tr>
<tr><th>37.7L</th><td>Length of a string literal </td><td>8191 bytes</td></tr>
<tr><th>37.8L</th><td>Length of a logical line in source </td><td>16383 bytes</td></tr>
<tr><th>37.9L</th><td>Number of macro definitions </td><td>8191</td></tr>
</table>
</blockquote>
<p>l_37_8.c does not become an execution program even if it is compiled. Only preprocessing is necessary, however, make an object file by doing cc -c l_37_8.c to compile. If preprocessing is successful, you can find out how long line the compiler proper can accept.</p>
<p>Scoring: 9. 1 point for each of 9 kinds of samples. Compiler proper testing is not included.</p>
<h4>q.1.2. Accuracy of diagnostic messages</h4>
<p>Though a diagnostic message is issued, it can be difficult to understand, too vague, or not obvious about the point of the problem. Some diagnostic messages are detailed while others may lack focus. A diagnostic message should not be simply "syntax error", but should show the reason for the error. The implementation should indicate the line and point out the token of the problem.</p>
<p>The error message of a mismatched #if section is desired to indicate the corresponding line. Otherwise, it is not possible to detect where the error resides.</p>
<p>Duplicate diagnostic messages for the same error are not desirable.</p>
<p>Scoring: 10.</p>
<h4>q.1.3. Accuracy of line number display</h4>
<p>It is not acceptable for the line number that a preprocessor passes to the compiler proper to be mismatched. This appears in a diagnostic message, however, it is treated as a separate item as a matter of convenience. This scoring is done whether line numbers are accurately displayed in the sample programs thus far. There are preprocessors that do not output the line number information. These are out of the question.</p>
<p>Scoring: 4.</p>
<h4>q.1.4. Runaway or abort</h4>
<p>Scoring: 20. Deduct points as below for implementations which runaway or abort in any of the samples in this Validation Suite. A "runaway" means that the program must be aborted by ^C, must be reset, or leaves a discrepancy in the file system. An "abort" means that the damage is not done though the program ends prematurely.</p>
<ol>
<li>0 point for a runaway in n_std.c (strictly conforming program.)<br>
<br>
<li>10 points (testing the part after where aborted causes another runaway, classify it as a "runway") if a processed is aborted in n_std.c.<br>
<br>
<li>10 points for a runaway in any of other samples.<br>
</ol>
<h3><a name="4.5.2" href="#toc.4.5.2">4.5.2. Options and extended functionalities</a></h3>
<h4>q.2.1. Specifying the include directory</h4>
<p>The so-called include directory where standard header files are located is fixed to one location in the most simple case. However, it is often the case that multiples exist and there are cases that users have to specify one. About user level header files, there is no problem in case those are in the current directory, however, the directory search rule is different depending on the implementation in case they are in another directory and they include another header file. In any case, it is inconvenient if the include directory cannot be specified by users explicitly by using options and environment variables (many implementations use -I option.) In addition, the option to exclude specified directories in the system is necessary in the case of searching multiple directories in sequence. The option to change the search rule itself justifies its existence.</p>
<p>Scoring: 4.</p>
<h4>q.2.2. Macro definition option</h4>
<p>It is valuable to have the option that allows an object-like macro to be defined at compilation rather than in source (many implementations use the -D option.) This makes it possible for the same source to generate objects of different specifications or to be compiled in different systems. In case of omitting a replacement list, there are some implementations defining 1 and others defining 0 piece of token, requiring a user to check documentation.</p>
<p>Some implementations have options to define the macro with arguments.</p>
<p>Scoring: 4.</p>
<h4>q.2.3. Macro cancellation option</h4>
<p>There should be an option to cancel implementation-specific built-in macros. There are the following types.</p>
<ol>
<li>2. The -U option which undefines a macro.<br>
<li>3. The option that removes implementation-specific built-in macros all at once.<br>
<li>4. The option that removes built-in macros prohibited in Standard C only ('unix' and others which do not start with _) all at once.<br>
</ol>
<p>Scoring: 2. 2 points with 1 or 2 option. 3 is not applicable as it is evaluated in d.1.5.</p>
<h4>q.2.4. Trigraphs option</h4>
<p>Though trigraphs are always used in the environment that needs them, they are hardly used in most environments since they are not needed. This should be enabled or disabled by an option at compilation.</p>
<p>Scoring: 2.</p>
<h4>q.2.5. Digraphs options</h4>
<p>Similarly, digraphs also should be enabled or disabled by an option at compilation.</p>
<p>Scoring: 2.</p>
<h4>q.2.6. Warning specifying option</h4>
<p>It will be more helpful to have warnings other than violations of syntax rules or constraints issued as much as possible, however, they can be annoying sometimes. The option to specify a warning level or one to enable or disable warnings by type is needed.</p>
<p>Scoring: 4.</p>
<h4>q.2.7. Other options</h4>
<p>There are other helpful options in preprocessing. Having too many options makes things complicated, however, there are some that are convenient. The option not to output the line number information (-P in most cases) is relatively common and it seems to be used for the purposes other than C language. There are some which output without deleting comments. Depending on command processors of OS, there are cases where implementations need to implement redirection of diagnostic messages. In addition, so-called one-pass compilers in which preprocessing is not an independent program should have an option to specify the output after preprocessing.</p>
<p>The option to identify C90 (C95), C99, and C++ is obviously necessary. Furthermore, the option to achieve the compatibility between C99 and C++ (making C++ preprocessing closer to C99's) is helpful.</p>
<p>There is an option to output source file dependency relations in order to generate a Makefile.</p>
<p>Scoring: 10.</p>
<h4>q.2.8. Extension by #pragma</h4>
<p>In Standard C, implementation specific directives are all implemented as sub-directives of #pragma. Preprocessing usually passes #pragma to a compiler as is and processes some of #pragma on its own. There are not many examples of the #pragma that preprocessing processes.</p>
<p>The header file with #pragma once written may be read only once even if #include to it is applied many times. This not only prevents multiple inclusion, but also is effective in improving processing speed. In case the whole header file is included without using #pragma once, for example, by</p>
<pre style="color:navy">
#ifndef __STDIO_H
#define __STDIO_H
..
#endif
</pre>
<p>some implementations automatically do not access this for a second time.</p>
<p>In <b>mcpp</b>, there is a #pragma directive which makes many header files combined as one file as a "pre-preprocess." In other words, this is the functionality to preprocess the header included by #include to output and add the #define directives which appeared there to the combined output. Including this is enough in compiling the original source. This way, the size of the header file gets smaller since comments, #if, and macro calls disappear. There is only one file to access. As a result, preprocessing speed is faster.</p>
<p>Some implementations are equipped with a header pre-compilation feature. This seems to have been introduced to process huge header files in C++, however, there is a tendency that the size of pre-compiled header gets larger than the total of original header files. The speed improvement cannot be seen very much in C, at least. The content of the pre-compiled header depends on the internal specification of compiler-proper. And the fact that it becomes a black box, which a user cannot see, is also a drawback.</p>
<p>In any case, above are all intended to speed up preprocessing and nothing else. Therefore, these features are not evaluated here, but rather in <a href=#q.3.1>q.3.1</a>.</p>
<p>Some implementations have #pragma which specifies the encoding of multi-byte characters. It is complete as a method of passing along the encoding of source files to a preprocessor or compiler.</p>
<p><b>mcpp</b> also has #pragma which traces preprocessing and outputs debug information. As no debugging can be performed for preprocessing using a regular debugger, this is an important feature that can be done only in a preprocessor. This feature is easier to use since debug points can be restricted by using #pragma rather than specifying an option.</p>
<p>Some implementations implement what is normally specified as an option at compilation by #pragma such as error and warning control specification. While #pragma has merits of no portability issues in as far as Standard C compliant implementations are concerned and of being able to specify the location on source, it has the demerit that the source files have to be rewritten when making changes to it. If this were to be implemented, it should be done so in addition to the implementation of the compilation option.</p>
<p>There are not very many other #pragma which are processed in preprocessing.</p>
<p>By the way, #pragma sub-directives are implementation-defined and have a possibility of name collision between implementations. Some device to avoid name collision is desirable. GCC 3 uses the name GCC as '#pragma GCC poison'. This is a good idea. <b>mcpp</b> has adopted this idea as '#pragma MCPP debug' since V.2.5.</p>
<p>Scoring: 10.</p>
<h4>q.2.9. Extended functionalities</h4>
<p>Although extended functionalities should be implemented as #pragma sub-directives, some preprocessors implement directives other than #pragma as proposals of new preprocessing specifications.</p>
<p>In Linux system headers, GCC / #include_next directive is used. Using of this directive means, however, organization of the system headers are unnecessarily complicated, and is not praiseworthy. GCC implements also non-Standard directives such as #warning, #assert, etc. whose needs are not high.</p>
<p>GCC / cc1 has a "traditional" mode option other than the standard behavioral mode. <b>mcpp</b> has options for various behavioral specifications. Those experiments also have some meanings.</p>
<p>Scoring: 6.</p>
<h3><a name="4.5.3" href="#toc.4.5.3">4.5.3. Efficiency and others</a></h3>
<h4><a name="q.3.1">q.3.1. Processing speed</a></h4>
<p>Though the accuracy of processing and diagnosis is the most important thing, the faster the speed is, the better.</p>
<p>The #pragma and option to improve speed are called out to find out the result of the speed.</p>
<p>Scoring: 20. The speed of the program that does nothing but copying input to output is set as 20 points. Comparative scoring is done speed relative to this benchmark. Refer to <a href="#6.2">6.2</a> for concrete measurements. Since the absolute speed varies depending on hardware, comparison should be done with same level of hardware. In addition, the speed of processing same program depends on the amount of standard headers to be read. Comparison should be done with <b>mcpp</b> as a point of reference.</p>
<p>In order to measure time, the time command (built-in command in bash or tcsh) is used for UNIX systems. On the Windows systems, there is a time command in bash if CygWIN is installed. Also, WindowsNT has a similar command called TimeThis in the "resource kit." On systems where these are not available, compile tool/clock_of.c for use (though it is rather inaccurate.) *1</p>
<p>Since preprocessing is usually done in an instant on recent hardware, it is hard to measure the time of preprocessing.
On Windows, the inclusion of Windows.h is a very heavy task, fortunately or unfortunately, and can be used to measure it.
On UNIX-like systems, we have to find some relatively heavy task from glibc sources or such.</p>
<p>Note:</p>
<p>*1 Some WindowsNT resource kit programs can be used on WindowsXP while others do not. TimeThis seems to be usable.</p>
<h4>q.3.2. Memory usage</h4>
<p>The smaller the memory usage is, the better it is. This is a serious problem especially where there is a strict limitation in memory usage.</p>
<p>As preprocessing is a part of compilation, the memory usage for overall compiler system actually becomes an issue. Since the compiler proper itself eats up more memory usually in case preprocessing is performed by an independent preprocessor, memory usage by a preprocessor shall not be a problem. However, in case there are many macro definitions and such, a preprocessor may consume more memory. Memory usage includes not only the size of a program, but also data area usage.</p>
<p>Scoring: 20.
There is no appropriate command to measure memory usage of a program.
On UNIX-like systems, we can roughly know the memory usage of cc1 or mcpp by 'top' command while doing 'make' of some large software such as glibc or firefox.
On Windows, memory usage of a preprocessor can be roughly known by 'task manager' about inclusion of Windows.h.</p>
<h4>q.3.3. Portability</h4>
<p>The portability of preprocessor source itself becomes an issue when it replaces a resident preprocessor of the compiler system or when it is being updated or customized. The following shall be subject to evaluation.</p>
<ol>
<li>Whether source is open (0 points if not open.)<br>
<li>Whether many compiler systems - OS's are supported.<br>
<li>What is the range of compiler systems - OS's for porting? Under what conditions?<br>
<li>Whether the source is easy to port.<br>
<li>Whether porting documents are provided.<br>
</ol>
<p>Scoring: 20. I only read one and a half source. I just took a glance at the rest. So, this scoring is just a guess.</p>
<h3><a name="4.5.4" href="#toc.4.5.4">4.5.4. Quality of Documents</a></h3>
<h4>q.4.1. Quality of documents</h4>
<p>In d.*, we saw only whether there are documents regarding "implementation-defined items" by Standard C. Here, we will also evaluate the quality of documents.</p>
<p>In addition to implementation-defined items, the following are necessary documents at minimum.</p>
<ol>
<li>Difference with Standard C.<br>
<li>Option specifications.<br>
<li>Meaning of diagnostic messages.<br>
</ol>
<p>In addition, having the description of the overall preprocessing specification including the Standard C would be of course much better.</p>
<p>Accuracy, readability, searchability, easiness of viewing and others become subjects for evaluation. The document for porting is included in the q.3.3 evaluation.</p>
<p>Scoring: 10.</p>
<br>
<h2><a name="4.6" href="#toc.4.6">4.6. C++ Preprocessing</a></h2>
<p>Nowadays, in most compiler systems C and C++ are provided together. In that case, the same preprocessor seems to be used in both C and C++. Since preprocessing is almost the same in both, it is not necessary to prepare separate preprocessors for each. However, it is not exactly the same in both.</p>
<p>If you compare C++ Standard with C90, the C++ preprocessing is the C90 preprocessing plus the specifications below.</p>
<ol>
<li>A character not included in the basic source character set is converted into a Unicode hexadecimal sequence in the \Uxxxxxxxx format in translation phase 1. And, this is converted again into an execution character set in the translation phase 5. *1<br>
<br>
<li>// to the end of the line is a comment. *2<br>
<br>
<li>Each of ::, .*, and -&gt;* is treated as one pp-token. *3<br>
<br>
<li>In the #if expression, 'true' and 'false' are evaluated as 1 and 0 respectively as a boolean literal. *4<br>
<br>
<li>11 kinds of identifier-like operators defined as a macro in the &lt;iso646.h&gt; standard header in ISO C : Amendment 1 (1995) are all tokens, not macros in C++ (pointless specification.)(*3) Similarly, 'new' and 'delete' are also operators. *5<br>
<br>
<li>The <tt>__cplusplus</tt> macro is predefined in 199711L. *6<br>
<br>
<li>Whether <tt>__STDC__</tt> is defined and how it should be defined if so are implementation-defined (On the contrary, it is undefined in C99 if <tt>__cplusplus</tt> is defined.) *6<br>
<br>
<li>Translation limits are extended to a large extent as below. However, these are just guidelines, not requirements. Implementations must explicitly document translation limits. *7<br>
<blockquote>
<table>
<tr><th>Length of a source logical line </th><td>65536 bytes</td></tr>
<tr><th>Length of a string literal, a character constant, and a header name</th><td>65536 bytes</td></tr>
<tr><th>Length of an identifier </th><td>1024 characters</td></tr>
<tr><th>#include nesting </th><td>256 levels</td></tr>
<tr><th>#if, #ifdef, and #ifndef nesting </th><td>256 levels</td></tr>
<tr><th>#if expression parentheses nesting </th><td>256 levels</td></tr>
<tr><th>Number of macro parameters </th><td>256</td></tr>
<tr><th>Number of definable macros </th><td>65536</td></tr>
</table>
</blockquote>
<li>The restriction (by Standards) in the length prior to '.' in a header name no longer exists. *8<br>
</ol>
<p>In C99, only the second of these are the same and the others are different. In C99, there occurred new differences due to additions such as p+ and P+ sequences in floating point numbers, official multi-byte character support in identifiers, variable macros, legitimization of empty arguments, macro expansion for the argument on the #pragma line, _Pragma() operator, #if expression evaluation in long long and wider, concatenation of a neighboring wide-character-string-literal and a character-string-literal as a wide-character-string-literal and others. UCN became a specification only in translation phase 5 in C99. The constraint on UCN is also slightly different. Though translation limits were also largely extended in C99 compared with C90, it was not as extreme as C++ Standard and there are differences here, too.</p>
<p>These may not seem to be too many differences but enough not to use the same preprocessing in C and C++. In addition, in C, C90 (C95) and C99 cannot use the same preprocessing.</p>
<p>Besides, predefining <tt>__STDC__</tt> in C++ is a cause of trouble and not desirable.</p>
<p>Although some implementations define <tt>__cplusplus</tt> using the -D option, it is inappropriate, as it becomes one of the user-defined macros.</p>
<p>Although whether each of ::, .*, and -&gt;* should be treated as one token is hardly an issue in preprocessing, handling it correctly is always the best.</p>
<p>Therefore, in order to share a preprocessor among C90 (C95), C99, and C++, a decent implementation seems to use a runtime option and change processing to accommodate differences above.</p>
<p>Please note that <b>mcpp</b> does not support the following two points among the specifications above since implementing them is too much burden for the value. I believe what I have is almost enough in practice.</p>
<ol>
<li>The conversion to UCN in translation phase 1 is not implemented. The C++ Standard states that UCN conversion does not necessarily have to be performed as long as the same outcome is obtained. Apart from the practical use, however, no same outcome would be obtained without conversions, strictly speaking. You can see this if you think about the character constant comparison between UCNs and multi-byte characters in the #if expression. In the stringizing by the # operator also, a problem occurs in a narrow sense. *9<br>
<br>
<li>Only up to 255 parameters at maximum for a macro.<br>
</ol>
<p>The conformance tests for C++ specific preprocessing specifications added to C90 preprocessing specifications are shown below.</p>
<p>In the implementations which do not recognize file names as C++ source unless they are in the format of *.cc, copy files to *.cc to test.</p>
<p>Samples more than what are in the test-l directory are not provided for translation limits. In C++, translation limits are only guidelines, so they are not subjects for scoring here. In addition, the length of header names will not be tested since it is OS dependent.</p>
<p>Note:</p>
<pre>
*1 C++ 2.1 Phases of translation
*2 C++ 2.7 Comments
*3 C++ 2.12 Operators and punctuators
*4 C++ 2.13.5 Boolean literals
</pre>
<p>In C99, bool, true, false, and __bool_true_false_are_defined are defined in &lt;stdbool.h&gt; as _Bool, 1, 0, and 1 respectively as a macro.</p>
<pre>
*5 C++ 2.5 Alternative tokens
*6 C++ 16.8 Predefined macro names
*7 C++ Annex B Implementation quantities
*8 C++ 16.2 Source file inclusion
</pre>
<p>In C90 6.8.2. only up to 6 characters to the left starting with '.' in a header name were guaranteed. 8 characters in C99 6.10.2. This restriction was removed in C++.</p>
<p>*9 In C99, whether extra \ is inserted when UCN is stringized by the # operator is implementation-defined. C++ does not have this specification.</p>
<p>If extra \ is added, the UCN no longer goes back to a multi-byte character. Therefore, it is a better implementation without an extra \. However, not having the extra \ is an "wrong" implementation in the C++ specification.</p>
<h4>4.5.n.ucn1. UCN recognition</h4>
<p>Scoring: 4.</p>
<h4>4.5.n.cnvucn. Conversion from a multi-byte character to UCN</h4>
<p>Scoring: 2.</p>
<h4>4.5.n.dslcom. // Comments</h4>
<p>Scoring: 4.</p>
<h4>4.5.n.bool. 'true' and 'false' are boolean literals</h4>
<p>Scoring: 2.</p>
<h4>4.5.n.token1. ::, .*, and -&gt;* are tokens</h4>
<p>Scoring: 2. Even if the test appears successful, it is invalid if token concatenation "succeeds" without any warning when processing it in C as well.</p>
<h4>4.5.n.token2. Alternative token for operators</h4>
<p>Scoring: 2.</p>
<h4>4.5.n.cplus. __cplusplus predefined macro</h4>
<p>Scoring: 4. 1 point for <tt>__cplusplus</tt> &lt; 199711L.</p>
<h4>4.5.u.cplus. #define, #undef __cplusplus</h4>
<p>Scoring: 1. 1 point if a diagnostic message such as a warning is issued.</p>
<h4>4.5.d.tlimit. Documents on the translation limits</h4>
<p>Scoring: 2.</p>
<br>
<h1><a name="5" href="#toc.5">5. Issues Around C Preprocessing</a></h1>
<p>Below are issues faced when a preprocessor is actually used other than those of C preprocessor Standard conformance level and quality.</p>
<br>
<h2><a name="5.1" href="#toc.5.1">5.1. Standard Header Files</a></h2>
<p>Samples in this Validation Suite include some standard headers. Without those headers correctly written, testing a preprocessor itself cannot be performed accurately.</p>
<p>The followings are prone to problems in practice in the standard header implementation.</p>
<h3><a name="5.1.1" href="#toc.5.1.1">5.1.1. General Rules</a></h3>
<p>The standard headers must not only include all function declarations, type definitions, and macro definitions, but also meet the conditions below.</p>
<ol>
<li>The identifier not specified by the Standard nor reserved must not be declared nor defined. The range declarable is determined for each standard header (range may be duplicate or shared between more than one standard header.) *1<br>
<br>
<li>Therefore, it is usually not acceptable for one standard header to include another one.<br>
<br>
<li>No matter in which order multiple standard headers are included, the results must be the same. *2<br>
<br>
<li>No matter how many times the same standard header is included, the result must be the same for other than &lt;assert.h&gt;. *2<br>
<br>
<li>Everything defined as an object-like macro which is expanded into an integer constant must be the #if expression. *3<br>
</ol>
<p>The range of the identifier reserved is specified by the Standard and other identifiers must be open to users. Since names starting with one or two '_' are reserved for some sort of usage, they can be used in standard headers by implementations (On the other hand, they must not be defined by users.)</p>
<p>This is a little constrained specification. No traditional names outside of Standards can be used in Standard C unless they are changed to the names starting with '_'. In POSIX which became a starting point for libraries and standard headers in Standard C, names outside of Standard C are also enclosed by:</p>
<pre style="color:navy">
#ifdef _POSIX_SOURCE
...
#endif
</pre>
<p>At least when this part is used, implementations are no longer Standard C.</p>
<p>Also, even if function names such as open(), creat(), close(), read(), write(), and lseek() do not appear in standard headers, implementing functions such as fopen(), fclose(), fread(), fgets(), fgetc(), fwrite(), fputs(), fputc(), fseek() etc. by using open() etc. violates user's name space indirectly. Therefore, dividing open() etc. in <tt>_POSIX_SOURCE</tt> or separating them in separate headers such as &lt;unistd.h&gt; on the surface is meaningless.</p>
<p>This type of "system call functions" should be changed to the names starting with '_' or ones that are essential in reality should be included in Standard C.</p>
<p>Although 2 is not clearly described in a specification, a standard header including another standard header usually results in the declaration of the name which the standard header cannot declare which is caught by 1. Each header including &lt;stddef.h&gt; is against the rule. To avoid this, non-standard header of a different name called &lt;sys/_defs.h&gt; or so can be provided and included by a standard header (also &lt;stddef.h&gt; itself.) And, names used there should all start with '_'. *4, *5</p>
<p>3 will not become an issue in reality.</p>
<p>4 had problems in old implementations but is complied by most current implementations.</p>
<p>The method of enclosing the whole standard header as below is common.</p>
<pre style="color:navy">
#ifndef _STDIO_H
#define _STDIO_H
...
#endif
</pre>
<p>In addition, there is a method of using extended directive such as #pragma once.</p>
<p>What becomes an issue in 5 is there are implementations where macros using sizeof or cast are written in standard headers. In Standard C, sizeof and cast are not allowed in the #if expression. Since sizeof and cast are available in the #if expression as well in the implementations actually using sizeof or cast in standard headers (Borland C 5.5), they must consider this as an extended specification.</p>
<p>As long as a user does not use sizeof and cast in the #if expression in his/her program, no portability problem or no other problems will arise. However, this preprocessing implementation is not an "extension" of Standard C, but rather a "deviation." That is because #if is processed in translation phase 4 in Standard C and no keyword exists in this phase. Keywords are recognized in phase 7 for the first time. In phase 4, names same as keywords are all treated as simple identifiers and identifiers not defined as macros are all evaluated as 0 on the #if line. So, sizeof (int) becomes 0 (0) and (int)0x8000 becomes (0)0x8000, causing a violation of syntax rule. Implementations must issue a diagnostic message for this. Not issuing a diagnostic message is not an "extension" but rather a "deviation" from Standard C. Recognizing only a part of keywords in phase 4 is a bit of a stretch as preprocessing logical configuration and confuses the meaning as "pre"-process phase in the compilation phase (translation phase 7.) *6</p>
<p>Note:</p>
<pre>
*1 C90 7.1.3 Reserved identifiers
C99 7.1.3 Reserved identifiers
</pre>
<pre>
*2 C90 7.1.2 standard headers
C99 7.1.2 standard headers
</pre>
<pre>
*3 C90 7.1.7 Use of library functions
C99 7.1.4 Use of library functions
</pre>
<p>*4 This method is used in the book below. This book has many points which serve as useful references to the implementation of compiler systems especially.</p>
<blockquote>
<p>P. J. Plauger "The Standard C Library", 1992, Prentice Hall</p>
</blockquote>
<p>*5 In the GNU glibc system, other standard headers such as &lt;stddef.h&gt; are read in by a standard header itself multiple times. However, what is defined at that time seems to be only the name in the range reserved. Although this is not against Standards, it is not a good method as it loses the readability of standard headers and it makes the maintenance difficult. It is better to use the file such as &lt;sys/_defs.h&gt;.</p>
<p>*6 Refer to <a href="#2.3">2.3</a> and <a href="#3.4.14">3.4.14</a>.</p>
<h3><a name="5.1.2" href="#toc.5.1.2">5.1.2. &lt;assert.h&gt;</a></h3>
<p>Next, we will look at each standard header. From what I see the standard headers attached to some implementations, the ones with most problems seem to be &lt;assert.h&gt; and &lt;limits.h&gt;. Although these 2 are the most easy headers, they tend to have errors in the implementation since they are newly defined as specifications in Standard C. The usage for these 2 is covered a little here.</p>
<p>At first, it is &lt;assert.h&gt;. *1, *2</p>
<p>Different from other standard headers, including this file many times is not the same. Depending on the NDEBUG is defined by a user changes the results every time the file is included. In other words, as needed, this header is used as below.</p>
<pre style="color:navy">
#undef NDEBUG
#include &lt;assert.h&gt;
assert( ...);
</pre>
<p>And, starting with area where debugging is complete, it becomes below.</p>
<pre style="color:navy">
#define NDEBUG
#include &lt;assert.h&gt;
assert( ...);
</pre>
<p>If NDEBUG is defined, assert(...); disappears after macro expansion even if it remains in source (even if ... is the expression with side effects, the side effects do not occur since the expression is not evaluated.)</p>
<p>In order for &lt;assert.h&gt; to be used like this, it must not be enclosed by:</p>
<pre style="color:navy">
#ifndef _ASSERT_H
#define _ASSERT_H
...
#endif
</pre>
<p>#pragma once must not be written, either.</p>
<p>Also, as you can see from this, assert() is a macro and changes its definition. &lt;assert.h&gt; must apply #undef assert and the assert macro must be redefined according to NDEBUG.</p>
<p>In the assert(expression) call, when NDEBUG is not defined, nothing happens if the expression is true. If it is false, that is reported in the standard error output. This report displays the expression as is (not expanded even if there is a macro) and also the file name of the source and the line number. This can be easily implemented as long as the # operator and the <tt>__FILE__</tt> and <tt>__LINE__</tt> macros are correctly implemented.</p>
<p>In reality, some of the old implementations do not implement the # operator correctly or do not implement &lt;assert.h&gt; correctly. There are many samples in this Validation Suite which includes &lt;assert.h&gt;, testing the preprocessor itself cannot be correctly performed if &lt;assert.h&gt; is not written correctly. Since it is easy to write &lt;assert.h&gt; correctly, it is better to rewrite the ones that are not right among the files in an implementation. The following is an example from C89 Rationale 4.2.1.1. In this as well, the correct result cannot be obtained if the # operator is not correctly implemented. However, it is a preprocessor problem and cannot be helped.</p>
<pre style="color:navy">
#undef assert
#ifndef NDEBUG
# define assert( ignore) ((void) 0)
#else
extern void __gripe( char *_Expr, char *_File, int _Line);
# define assert( expr) \\
((expr) ? (void)0 : __gripe( #expr, __FILE__, __LINE__))
#endif
</pre>
<p>The __gripe() function can be written as below (the name, __gripe, can be anything as long as it starts with '_'.)</p>
<pre style="color:navy">
#include &lt;stdio.h&gt;
#include &lt;stdlib.h&gt;
void __gripe( char *_Expr, char *_File, int _Line)
{
fprintf( stderr, "Assertion failed: %s, file %s, line %d\n",
_Expr, _File, _Line);
abort();
}
</pre>
<p>Some implementations write fprintf(), fputs(), or abort() directly in &lt;assert.h&gt; without using __gripe(). That is also acceptable, however, it requires declarations for these functions. The FILE and stderr declarations are also necessary. However, it is quite complicated since &lt;stdio.h&gt; cannot be included. There will be no mistake if a separate function is implemented.</p>
<p>This is not so significant, but a duplicate string literal is generated every call in case all are implemented by macros. If implementations do not perform optimization by merging duplicate string literals into one, this is not wise in terms of the code size.</p>
<p>Note:</p>
<pre>
*1 C90 7.2 Diagnostics &lt;assert.h&gt;
C99 7.2 Diagnostics &lt;assert.h&gt;
</pre>
<p>*2 Starting C99, the assert() macro started displaying from which function a macro was called. The internal identifier, __func__, is defined for this type of purpose.</p>
<h3><a name="5.1.3" href="#toc.5.1.3">5.1.3. &lt;limits.h&gt;</a></h3>
<p>This standard header is where macros representing the range of the integer type and others are written. These macros must be written so that their values match specifications and also the following conditions must be met. *1</p>
<ol>
<li>Must be an integer constant expression which can be used in the #if directive.<br>
<br>
<li>Must be the expression of the same type as the object of the corresponding type after integer promotion.<br>
</ol>
<p>There are implementations that use cast. We discussed that sizeof and cast in the #if expression are not in the range of Standard C in <a href="#5.1">5.1</a>. First of all, the meaning that &lt;limits.h&gt; was newly specified is in that a preprocessor does not have to perform query regarding the execution environment such as cast and sizeof.</p>
<p>For example, instead of</p>
<pre style="color:navy">
#if (int)0x8000 &lt; 0
</pre>
<p>and</p>
<pre style="color:navy">
#if sizeof (int) &lt; 4
</pre>
<p>the following is used:</p>
<pre style="color:navy">
#include &lt;limits.h&gt;
#define VALMAX ...
#if INT_MAX &lt; VALMAX
</pre>
<p>In #if, cast and sizeof are not necessary if &lt;limits.h&gt; macros are used.</p>
<p>Examples where the &lt;limits.h&gt; macros are wrong in the type can be seen from time to time. Those do not come from the preprocessor specification, but it seems that the person writing &lt;limits.h&gt; forgets 2 above and integer promotion, the usual arithmetic conversion rule, or the evaluation rule for integer constant tokens.</p>
<p>For example, there is a definition as below.</p>
<pre style="color:navy">
#define UCHAR_MAX 255U
</pre>
<p>The unsigned char values are all in the int range (if <tt>CHAR_BIT</tt> is 8) and the data object values in the unsigned char type becomes int by integer promotion. Therefore, <tt>UCHAR_MAX</tt> also must be evaluated as int. However, it becomes unsigned int in 255U. This must be:</p>
<pre style="color:navy">
#define UCHAR_MAX 255
</pre>
<p>Although either way does not seem to have any issues in practice, it is not necessarily so. Operations including unsigned type cause "usual arithmetic conversion" and force the conversion from signed type to unsigned of the same size. Therefore, the result of value comparison varies.</p>
<p>This is easy to see in the following:</p>
<pre style="color:navy">
assert( -255 &lt; UCHAR_MAX);
</pre>
<p>This mistake is related to the circumstance that the integer promotion and usual arithmetic conversion rules were changed in Standard C from the ones adopted in many conventional implementations. The unsigned char, unsigned short, and unsigned long types were not in K&amp;R 1st, but were implemented in many implementations later. In addition, in most of those implementations, unsigned was always converted into unsigned.</p>
<p>In the integer promotion in Standard C, however, unsigned char and unsigned short are promoted into int as long as all values stay in the int range and they are promoted into unsigned int otherwise. Also, in the usual arithmetic conversion between unsigned int and long, all values of unsigned int are converted into long if they are in the long range and into unsigned long otherwise. This is called the change from "unsigned preserving rules" to "value preserving rules." The reason for the specification is that this is supposed to be more predictable intuitively. A caution is necessary for this rule in &lt;limits.h&gt;. *2</p>
<p>In all examples below, short is 16 bit and long is 32 bit. The value of <tt>USHRT_MAX</tt> is 65535, but how to write depends on if int is 16 bit or 32 bit.</p>
<pre style="color:navy">
#define USHRT_MAX 65535U /* if int is 16 bit */
#define USHRT_MAX 65535 /* if int is 32 bit */
</pre>
<p>Since unsigned short is not in the int range if int is 16 bit, it is promoted into unsigned int. Therefore, <tt>USHRT_MAX</tt> also must be evaluated as unsigned int. In 65535, it is evaluated as long. The suffix, 'U', is necessary. On the other hand, since unsigned short values are all in the int range if int is 32 bit, they are promoted into int. Therefore, <tt>USHRT_MAX</tt> also must be evaluated as int. 'U' must not be attached. However, there is an example opposite of this.</p>
<pre style="color:navy">
#define USHRT_MAX 0xFFFF
</pre>
<p>In this example, correct evaluation will be performed whether int is 16 bit or 32 bit. In Standard C, octal or hexadecimal integer constant tokens without U, u, L, and l suffix are evaluated in the type which can express the value non-negative in the order of int, unsigned int, long, and unsigned long. In other words, 0xFFFF is evaluated as 65535 of unsigned int if int is 16 bit and 65535 of int if int is 32 bit. On the other hand, decimal integer tokens without a suffix are evaluated in the order of int, long, and unsigned long. 65535 is evaluated as long if int is 16 bit and as int if int is 32 bit. *3</p>
<p>C99 added long long/unsigned long long. It also added the _Bool type which has only the 0 or 1 value. Other types of integers became implementable as well. Rules for integer promotion were extended and the integer constant tokens that cannot be expressed as unsigned long are evaluated as long long/unsigned long long.</p>
<p>In accordance with the increased integer types and the acceptance of the implementation-defined integer types, the size relations of types became confusing. Therefore, the concept of integer conversion rank was introduced. This concept is a little complex, but there is no need to worry in practice. In standard integer types, the size relations of the rank are as below.</p>
<p>long long &gt; long &gt; int &gt; short &gt; char &gt; _Bool</p>
<p>Here, the point is that the rank size for the implementations of the same size, for example, long and int which are both 32 bit, are distinguished. *4, *5</p>
<p>Note:</p>
<pre>
*1 C90 5.2.4.2.1 Sizes of integral types &lt;limits.h&gt;
C99 5.2.4.2.1 Sizes of integer types &lt;limits.h&gt;
*2 C90 6.2.1 Arithmetic operands
C99 6.3.1 Arithmetic operands
*3 C90 6.1.3.2 Integer constants
C99 6.4.4.1 Integer constants
*4 C99 6.4.4.1 Integer constants
</pre>
<p>*5 C99 added standard headers called &lt;stdint.h&gt; and &lt;inttypes.h&gt; in order to absorb the differences in integer types by implementations. These typedef some type names other than the long and short names since the number of integer types increased due to the arrival of 64 bit systems and the corresponding relations became confusing. However, there are 26 kinds of these type names, 42 kinds of macros representing the maximum and minimum values corresponding to these, 56 kinds of macros converted into the format specifier of corresponding fprintf(), and 56 kinds of macros converted into the format specifier of fscanf() similarly. Although there is no much load on implementations, too much complexity gives the impression of terminal symptoms.</p>
<h4>5.1.3.1. INT_MIN</h4>
<p>Among all macros in &lt;limits.h&gt;, the most confusing ones are <tt>INT_MIN</tt> and <tt>LONG_MIN</tt> in the system with the internal representation of 2's complement. Especially, the <tt>INT_MIN</tt> in the implementation where int is 16 bit and long is 32 bit shows all the problems above. These are specially covered in separate sections.</p>
<p>In this case, it is understood that the range of int is [-32768,32767]. Additionally, there are no problems by having <tt>INT_MAX</tt> as 32767 or 0x7FFF in any implementation. However, I see an example where <tt>INT_MIN</tt> is defined as below.</p>
<pre style="color:navy">
#define INT_MIN (-32767)
</pre>
<p>Why is this type of definition different from the reality?</p>
<p>On the other hand, there are no implementations with this type of definition, as might be expected.</p>
<pre style="color:navy">
#define INT_MIN (-32768)
</pre>
<p>-32768 consists of 2 tokens, - and 32768. And, 32768 is not in the range which can be expressed in int. So, this is evaluated as long. Therefore, -32768 becomes the meaning of - (long) 32768.</p>
<p>Some make a definition like this:</p>
<pre style="color:navy">
#define INT_MIN ((int)0x8000)
</pre>
<p>No comment is repeated for the definition using cast. It is also invalid since 0x8000 only becomes the meaning of (unsigned) 32768.</p>
<p>Then, how can the definition be in order to make an evaluation as (int) -32768 without casting?</p>
<pre style="color:navy">
#define INT_MIN (-32767-1)
</pre>
<p>This is fine. 32767 can be <tt>INT_MAX</tt> or 0x7FFF. This definition has a subtraction operation, but it is not an issue (unary - is an operator in the first place.) *1, *2</p>
<pre style="color:navy">
#define INT_MIN (~INT_MAX)
#define INT_MIN (1&lt;&lt;15)
</pre>
<p>These are also correct definitions.</p>
<pre style="color:navy">
#define INT_MIN (-32767)
</pre>
<p>I can imagine that this gave up defining a correct value since the idea of operation did not occur.</p>
<p>Then, is the definition of -32767 wrong or correct?</p>
<p>The bottom line is that this is wrong.</p>
<p><tt>INT_MIN</tt> is defined as a macro representing the minimum value of int. If <tt>INT_MIN</tt> is -32767, what does this mean? And, what is <tt>INT_MIN</tt>-1 at all? Or, what are ~<tt>INT_MAX</tt> and 1&lt;&lt;15?</p>
<p>Regarding the <tt>INT_MIN</tt>-1 in this case, there seems to be thought of as the bit pattern representing out of range such as "NaN" in floating point operation.</p>
<p>However, compared with the Standard C specification regarding integer type, this interpretation has no basis. First, the result of bit operations on integer types is undefined in case the value of op2 is negative or the number of bit for op1 type and above where op1 &lt;&lt; op2 or op1 &gt;&gt; op2, and not undefined, returning all unique values of integer type, otherwise. The result of ~op is int if op is int and the results of op1 &amp; op2, op1 | op2, op1 &lt;&lt; op2, and op1 &gt;&gt; op2 are int if both op1 and op2 are int. Therefore, the results of ~<tt>INT_MAX</tt> and 1&lt;&lt;15 are both int. You may think 1&lt;&lt;15 will overflow, but it is not so. Since the bit operation returns the value corresponding to the bit pattern as result of the bit operation, overflow cannot occur.</p>
<p>In C, the integer type operations are defined well in general. There are extremely few undefined areas. Especially, the relationship between a bit pattern and a value corresponds one-on-one completely except when 2 bit patterns exist for 0 in the internal representations of 1's complement and sign+absolute value. This is consistent from K&amp;R 1st to Standard C. C has no way to write a bit pattern itself and "Not-a-Number" can be written only as (-32767-1) etc. This is an int value itself as you can see. C89 Rationale mentions some grounds and made clear there is no room for bit patterns representing an "invalid integer" or "illegal integer" in the integer type. *3, *4</p>
<p>In the internal representation of 2's complement, ~<tt>INT_MAX</tt> is the value of <tt>INT_MIN</tt> and I must say that the definition bigger than that is wrong.</p>
<p>Note:</p>
<p>*1 I saw this definition in "The Standard C Library" by P. J. Plauger for the first time. This style of limits.h is getting popular in recent implementations.</p>
<p>However, limits.h in this book also contains a mistake. The definitions for the compiler system with 16 bit int and 32 bit long are as below.</p>
<pre style="color:navy">
#define UINT_MAX 65535
#define USHRT_MAX 65535
</pre>
<p>These will evaluate to long. The correct definitions are:</p>
<pre style="color:navy">
#define UINT_MAX 65535U
#define USHRT_MAX 65535U
</pre>
<p>*2 In recent compiler systems, *_MIN is typically defined in the form of (-*_MAX - 1) and there are few mistakes though they are still found occasionally. Vc7/include/limits.h and Vc7/crt/src/include/limits.h in Visual C++ 2003 contains:</p>
<pre style="color:navy">
#define LLONG_MIN 0x8000000000000000
</pre>
<p>0x8000000000000000 evaluates to unsigned long long. Since this type has the highest rank, the result of integer promotion has the same type. It never becomes a negative value. Therefore,</p>
<pre style="color:navy">
#if LLONG_MAX &gt; LLONG_MIN
</pre>
<p>does not turn out as expected.</p>
<p><tt>LLONG_MIN</tt> in include/limits.h of LCC-Win32 2003-08, 2006-03 is as below.</p>
<pre style="color:navy">
#define LLONG_MIN -9223372036854775808LL
</pre>
<p>9223372036854775808LL is a violation of constraints as this token value overflows the range of signed long long. If <tt>LLONG_MIN</tt> is defined as:</p>
<pre style="color:navy">
#define LLONG_MIN -9223372036854775808LLU
</pre>
<p>9223372036854775808LLU becomes unsigned long long. Using the unary - operator on an unsigned type does not change the result type, however, the result becomes a value which cannot be expressed in unsigned long long and ends up undefined.</p>
<p>In Visual C++ 2003 and LCC-Win32, all other *_MIN definitions are (-*_MAX - 1), but why is only <tt>LLONG_MIN</tt> wrong? If it is defined as below, there will be no problem.</p>
<pre style="color:navy">
#define LLONG_MIN (-LLONG_MAX - 1LL)
</pre>
<p>In Visual C++ 2005, this definition was revised correctly.</p>
<pre>
*3 C89 Rationale 3.1.2.5 Types
C99 Rationale 6.2.6.2 Integer types
</pre>
<p>*4 In C99, the handling of the specific bit pattern as "trap representation" which causes an exception is allowed in implementations.</p>
<p>I do not know what sort of implementations fall under this specification in reality.</p>
<h3><a name="5.1.4" href="#toc.5.1.4">5.1.4. &lt;iso646.h&gt;</a></h3>
<p>ISO C 9899:1990/ Amendment 1 added the standard header called iso646.h. This provides the operators including &amp;, |, ~, ^, or ! with the replacement spelling expressed only in invariant character set in ISO 646. Replacement spelling is provided for |, ~, and ^ in trigraphs as well, however, trigraphs lack in readability. Alternatively, iso646.h defines 11 types of operators in macros in the token unit.</p>
<p>This implementation is very easy and the following example is enough. Since macro expansion is performed in preprocessing, there is no trouble for implementations. *1</p>
<pre style="color:navy">
/* iso646.h ISO 9899:1990 / Amendment 1 */
#ifndef _ISO646_H
#define _ISO646_H
#define and &amp;&amp;
#define and_eq &amp;=
#define bitand &amp;
#define bitor |
#define compl ~
#define not !
#define not_eq !=
#define or ||
#define or_eq |=
#define xor ^
#define xor_eq ^=
#endif
</pre>
<p>Note:</p>
<p>*1 In the C++ Standard, these identifier-like operators are specified as operator-tokens rather than macros. This is a troublesome and meaningless specification for implementations.</p>
<br>
<h1><a name="6" href="#toc.6">6. Preprocessor Test Results</a></h1>
<h2><a name="6.1" href="#toc.6.1">6.1. Preprocessors Tested</a></h2>
<p>Compiler systems tested and execution methods are as below. Compiler systems are sorted in the order of release time.</p>
<p>Runtime options slightly vary in each of C95 (C90), C99, and C++98.</p>
<p>If there are problems in &lt;assert.h&gt; and &lt;limits.h&gt;, testing was done after correctly rewriting them.</p>
<pre>
Number: OS / Compiler System / Execution program (version)
Runtime option
Comment
1 : Linux / / DECUS cpp
C95: cpp
</pre>
<blockquote>
DECUS cpp original version by Martin Minow (June 1985.) It was ported to some systems such as various DEC systems, UNIX, and MS-DOS at that time, but what used in this test was modified by kmatsui and compiled on Linux / GCC. Macros were rewritten so that translation limits clear as many specifications as possible.
</blockquote>
<pre>
2 : FreeBSD 2.2.7 / GCC V.2.7.2.1 / cpp (V.2.0)
GO32 / DJGPP V.1.12 / cpp (V.2.0)
WIN32 / BC 4.0 / cpp (V.2.0)
MS-DOS / BC 4.0, TC 2.0 / cpp (V.2.0)
MS-DOS / LSI C-86 V.3.3 / cpp (V.2.0)
OS-9/6x09 / Microware C/09 / cpp (V.2.0)
C95: cpp -23 (-S1 -S199409L) -W15
gcc -ansi -Wp,-2,-W15
C99: cpp -23 (-S1) -S199901L -W15
C++: cpp -23+ -S199711L -W15
</pre>
<blockquote>
Open-source-software by kmatsui (August, 1998.) Called <b>mcpp</b>. Rewrite of DECUS cpp.
I have compiled with GCC on Linux and used the executable for this test.
</blockquote>
<pre>
3 : WIN32 / Borland C++ V.5.5J / cpp32 (August, 2000)
C95: cpp32 -A -w
bcc32 -A -w
C99: cpp32 -A -w
C++: cpp32 -A -w
</pre>
<blockquote>
Trigraphs are not processed by cpp32 nor bcc32. Instead, a conversion program called trigraph.exe is provided.
It just set up an alibi called "Standard conformance." In Borland C, I used this program to convert trigraphs in advance to test (lenient testing.) This trigraph.exe, however, processes even line splicing by &lt;backslash&gt;&lt;newline&gt;.
Therefore, the line number is out of alignment (deduct scores in q.1.2.)
</blockquote>
<pre>
4 : Linux, CygWIN / GCC V.2.95.3 (March, 2001) / cpp0
C95: cpp0 -D__STRICT_ANSI__ -std=c89 -$ -pedantic -Wall
gcc -ansi -pedantic -Wall
C99: cpp0 -std=c9x -$ -Wall
C++: g++ -E -trigraphs -$ -Wall
</pre>
<blockquote>
Since GCC is portable source, the person who ported it should prepare the specification for what has been ported to a specific system. However, no such document is provided. Only GNU cpp.info exists as a cpp document.
</blockquote>
<pre>
5 : Linux / GCC V.3.2 (August, 2002) / cpp0
C95: cpp0 -D__STRICT_ANSI__ -std=iso9899:199409 -$ -pedantic
-Wall
gcc -std=iso9899:199409 -pedantic -Wall
C99: cpp0 -std=c99 -$ -Wall
C++: g++ -E -trigraphs -$ -Wall
</pre>
<blockquote>
Compiled from the source by kmatsui. Configured with --enable-c-mbchar option.
</blockquote>
<pre>
6 : Linux / / ucpp (V.1.3)
C95: ucpp -wa -c90
C99: ucpp -wa
</pre>
<blockquote>
Open-source-software by Thomas Pornin (January, 2003.) A portable compiler-independent preprocessor. I compiled it on Linux by GCC.
</blockquote>
<pre>
7 : WIN32 / Visual C++ 2003 / cl
C95: cl -Za -E -Wall -Tc
C99: cl -E -Wall -Tc
C++: cl -E -Wall -Tp
</pre>
<blockquote>
Since the -E option does not process comments and &lt;backslash&gt;&lt;newline&gt; properly, a compilation testing is used together (April, 2003.)
</blockquote>
<pre>
8 : WIN32 / LCC-Win32 2003-08 / lcc
C95: lcc -A -E
lcc -A
C99: lcc -A -E
C++: lcc -A -E
</pre>
<blockquote>
An integrated development environment which Jacob Navia wrote based on open-source-software by C. W. Fraser &amp; Dave Hanson (August, 2003.) The preprocessing part is based on source originally written for Plan9 by Dennis Ritchie.
</blockquote>
<pre>
9 : WIN32, Linux, etc. / / wave (V.1.0)
C95: wave
C99: wave --c99
C++: wave
</pre>
<blockquote>
Open-source-software by Hartmut Kaiser (January, 2004.) A portable compiler-independent preprocessor. Implemented using the C++ library called "Boost preprocessor library" by Paul Mensonides et al. Used a binary on Windows made by the author of wave.
</blockquote>
<pre>
10 : FreeBSD, Linux, CygWIN / GCC 2.95, 3.2
WIN32, MS-DOS / Visual C 2003, BCC, etc. / mcpp_std (V.2.4)
C95: mcpp_std -23 (-S1 -V199409L) -W31
gcc -ansi -Wp,-2,-W31
C99: mcpp_std -23 (-S1) -V199901L -W31
C++: mcpp_std -23+ -V199711L -W31
</pre>
<blockquote>
<b>mcpp</b> V.2.4 (February, 2004), in Standard mode. Compiled with Linux / GCC.
</blockquote>
<pre>
11 : Linux / GCC V.3.4.3 (November, 2004) / cc1, cc1plus
C95: gcc -E -std=iso9899:199409 -pedantic -Wall
C99: gcc -E -std=c99 -$ -Wall
C++: g++ -E -std=c++98 -$ -Wall
</pre>
<pre>
12 : WIN32 / Visual C++ 2005 / cl
C95: cl -Za -E -Wall -Tc
C99: cl -E -Wall -Tc
C++: cl -E -Wall -Tp
</pre>
<blockquote>
Since the -E option does not process comments and &lt;backslash&gt;&lt;newline&gt; properly, a compilation testing is used together. (September, 2005.)
</blockquote>
<pre>
13 : WIN32 / LCC-Win32 2006-03 / lcc
C95: lcc -A -E
lcc -A
C99: lcc -A -E
C++: lcc -A -E
</pre>
<blockquote>
LCC-Win32 2006-03 (March, 2006.)
</blockquote>
<pre>
14 : Linux / GCC V.4.1.1 (May, 2006) / cc1, cc1plus
C95: gcc -E -std=iso9899:199409 -pedantic -Wall
C99: gcc -E -std=c99 -$ -Wall
C++: g++ -E -std=c++98 -$ -Wall
15 : WIN32 / Visual C++ 2008 / cl
C95: cl -Za -E -Wall -Tc
C99: cl -E -Wall -Tc
C++: cl -E -Wall -Tp
</pre>
<blockquote>
Since the -E option does not process comments and &lt;backslash&gt;&lt;newline&gt; properly, a compilation testing is used together. (December, 2007.)
</blockquote>
<pre>
16: Linux, WIN32, etc. / / wave (V.2.0)
C95: wave (--c99)
C99: wave --c99
C++: wave
</pre>
<blockquote>
wave V.2.0 (2008/08)<29><>
Compiled by kmatsui on Linux/GCC and Windows/Visual C++ from the source contained in Boost C++ Library V.1.36.0 with its default settings.
Used with configuration files for header files of GCC and Visual C++.
</blockquote>
<pre>
17: FreeBSD, Linux, Mac OS X, CygWIN, MinGW / GCC 2.95-4.1
WIN32 / Visual C 2003-2008, BCC, LCC-Win32 / mcpp (V.2.7.2)
C95: mcpp -23 (-S1 -V199409L) -W31
gcc -ansi -Wp,-2,-W31,-fno-dollars-in-identifiers
C99: mcpp -23 (-S1) -V199901L -W31
C++: mcpp -23+ -V199711L -W31
</pre>
<blockquote>
<b>mcpp</b> V.2.7.2 (2008/11)<29><>
</blockquote>
<br>
<h2><a name="6.2" href="#toc.6.2">6.2. Lists of Marks</a></h2>
<pre>
D M B G G u V L W M G V L G V W M
E C C C C c C C a C C C C C C A C
C P C C C p 2 C v P C 2 C C 2 V P
U P 5 2 3 p 0 0 e P 3 0 0 4 0 E P
S 2 5 9 2 1 0 3 1 2 4 0 6 1 0 2 2
C 0 C 5 3 3 0 0 4 3 5 0 1 8 0 7
P P 3 8 3 2
P P
max 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17
[K&R: Processing of sources conforming to K&R and C90] (31 items)
n.2.1 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4
n.2.2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2
n.2.3 2 0 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2
n.6.1 10 10 10 10 10 10 10 10 10 4 10 10 10 10 10 10 10 10
n.7.2 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4
n.10.2 6 0 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6
n.12.3 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4
n.12.4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4
n.12.5 2 0 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2
n.12.7 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4
n.13.1 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2
n.13.2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2
n.13.3 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4
n.13.4 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2
n.13.7 6 6 6 4 6 6 6 4 6 0 6 6 4 4 6 4 6 6
n.13.8 2 0 2 2 2 2 2 2 2 0 2 2 2 2 2 2 2 2
n.13.9 2 2 2 2 2 2 2 0 2 0 2 2 0 2 2 0 2 2
n.13.10 2 2 2 2 2 2 2 0 0 2 2 2 2 0 2 2 2 2
n.13.11 2 0 2 2 2 2 2 0 0 0 2 2 2 0 2 2 2 2
n.13.12 2 0 2 2 2 2 2 2 0 0 2 2 2 0 2 2 2 2
n.15.1 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6
n.15.2 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6
n.18.1 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30
n.18.2 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20
n.18.3 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10
n.27.1 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6
n.27.2 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4
n.29.1 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10
n.32.1 2 2 2 2 2 2 2 2 2 0 2 2 2 2 2 2 2 2
i.32.3 2 2 2 2 2 2 2 2 2 0 2 2 2 2 2 2 2 2
i.35.1 2 2 2 2 2 2 2 0 0 0 2 2 0 0 2 0 1 2
stotal 166 150 166 164 166 166 166 156 158 140 166 166 160 156 166 160 165 166
[C90: Processing of strictly conforming sources] (76 items)
n.1.1 6 0 6 6 6 6 6 6 6 0 6 6 6 6 6 6 6 6
n.1.2 2 0 2 2 2 2 2 2 2 0 2 2 2 2 2 2 2 2
n.1.3 2 0 2 2 2 2 2 2 2 0 2 2 2 2 2 2 2 2
n.2.4 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2
n.2.5 2 0 2 2 2 2 2 2 2 0 2 2 2 2 2 2 2 2
n.3.1 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6
n.3.3 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4
n.3.4 2 0 2 0 2 2 0 2 2 2 2 2 2 2 2 2 2 2
n.4.1 6 0 6 0 6 6 6 6 0 0 6 6 6 0 6 6 6 6
n.4.2 2 0 2 0 2 2 2 2 0 0 2 2 2 0 2 2 2 2
n.5.1 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6
n.6.2 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6
n.6.3 2 0 2 0 2 2 2 2 2 2 2 2 2 2 2 2 2 2
n.7.1 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6
n.7.3 4 0 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4
n.8.1 8 0 8 8 8 8 8 8 8 8 8 8 8 8 8 8 2 8
n.8.2 2 0 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2
n.9.1 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10
n.10.1 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10
n.11.1 8 8 8 8 8 8 8 8 8 8 8 8 8 8 8 8 8 8
n.11.2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2
n.12.1 6 0 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6
n.12.2 4 0 4 4 4 4 4 4 4 0 4 4 4 4 4 4 4 4
n.12.6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6
n.13.5 2 2 2 2 2 2 2 2 2 0 2 2 2 2 2 2 2 2
n.13.6 6 0 6 6 6 6 4 6 4 0 6 6 4 4 6 4 6 6
n.13.13 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4
n.13.14 2 0 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2
n.19.1 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4
n.19.2 4 2 4 4 4 4 4 4 4 2 4 4 4 4 4 4 4 4
n.20.1 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6
n.21.1 4 0 4 0 4 4 4 0 4 4 4 4 0 4 4 0 4 4
n.21.2 2 0 2 0 2 2 2 0 2 2 2 2 0 2 2 0 2 2
n.22.1 4 0 4 0 4 4 4 4 4 0 4 4 4 4 4 4 4 4
n.22.2 2 0 2 0 2 2 2 2 2 0 2 2 2 2 2 2 2 2
n.22.3 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 0 2
n.23.1 6 2 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6
n.23.2 2 2 2 2 2 2 2 2 2 0 2 2 2 2 2 2 2 2
n.24.1 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6
n.24.2 4 0 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4
n.24.3 6 0 6 0 6 6 6 6 6 6 6 6 6 6 6 6 6 6
n.24.4 2 0 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2
n.24.5 2 0 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2
n.25.1 4 2 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4
n.25.2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2
n.25.3 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2
n.25.4 6 0 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6
n.25.5 4 0 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4
n.26.1 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2
n.26.2 2 0 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2
n.26.3 2 0 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2
n.26.4 2 0 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2
n.26.5 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2
n.27.3 2 0 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2
n.27.4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 2 4
n.27.5 2 2 2 2 2 2 2 0 2 0 2 2 0 2 2 0 0 2
n.27.6 2 0 0 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2
n.28.1 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4
n.28.2 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4
n.28.3 4 0 4 4 4 4 2 4 4 4 4 4 4 4 4 4 4 4
n.28.4 4 0 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4
n.28.5 4 0 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4
n.28.6 4 0 4 0 4 4 2 0 0 4 4 4 0 0 4 0 4 4
n.28.7 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4
n.29.2 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4
n.30.1 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6
n.32.2 2 2 2 2 2 2 2 2 2 0 2 2 2 2 2 2 2 2
n.37.1 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2
n.37.2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2
n.37.3 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4
n.37.4 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2
n.37.5 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4
n.37.6 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2
n.37.7 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2
n.37.8 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2
n.37.9 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4
stotal 286 160 284 252 286 286 278 274 272 240 286 286 272 272 286 272 274 286
[C90: Processing of implementation defined portions] (1 item)
i.32.4 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2
stotal 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2
[C90: Diagnosing of violation of syntax rule or constraint] (50 items)
e.4.3 2 2 2 1 2 2 2 2 2 2 2 2 2 2 2 2 2 2
e.7.4 2 0 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2
e.12.8 2 0 2 2 2 2 2 2 0 2 2 2 2 0 2 2 2 2
e.14.1 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2
e.14.2 4 2 4 2 4 4 2 2 4 4 4 4 4 4 4 4 4 4
e.14.3 2 2 2 2 2 2 1 2 2 2 2 2 2 2 2 2 2 2
e.14.4 2 2 2 2 2 2 1 2 2 2 2 2 2 2 2 2 2 2
e.14.5 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2
e.14.6 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2
e.14.7 2 0 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2
e.14.8 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2
e.14.9 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2
e.14.10 4 0 4 2 0 0 0 0 0 0 4 0 0 0 0 0 4 4
e.15.3 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2
e.15.4 2 2 2 1 2 2 2 1 2 2 2 2 2 2 2 2 2 2
e.15.5 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2
e.16.1 2 2 2 1 2 2 2 1 2 2 2 2 2 2 2 2 2 2
e.16.2 2 2 2 1 2 2 2 1 2 2 2 2 2 2 2 2 2 2
e.17.1 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2
e.17.2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2
e.17.3 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2
e.17.4 2 2 2 2 2 2 2 2 2 0 2 2 2 2 2 2 2 2
e.17.5 2 0 2 2 2 2 2 2 2 0 2 2 2 2 2 2 2 2
e.17.6 2 0 2 2 2 2 2 2 2 0 2 2 2 2 2 2 2 2
e.17.7 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2
e.18.4 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2
e.18.5 2 2 2 2 0 2 2 2 2 2 2 2 2 2 2 2 2 2
e.18.6 2 0 2 2 2 2 2 2 2 0 2 2 2 2 2 2 2 2
e.18.7 2 0 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2
e.18.8 2 2 2 2 2 2 2 2 2 0 2 2 2 2 2 2 2 2
e.18.9 2 0 2 0 2 2 2 2 0 0 2 0 2 0 2 2 0 2
e.19.3 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4
e.19.4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4
e.19.5 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4
e.19.6 2 0 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2
e.19.7 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2
e.23.3 2 0 2 2 2 2 2 2 0 0 2 2 2 0 2 2 0 2
e.23.4 2 2 2 2 2 2 2 2 0 0 2 2 2 0 2 2 0 2
e.24.6 2 2 2 2 2 2 2 2 0 0 2 2 2 0 2 2 0 2
e.25.6 4 0 4 0 4 4 4 4 4 4 4 4 4 4 4 4 4 4
e.27.7 2 0 2 2 2 2 2 0 2 2 2 2 0 2 2 0 2 2
e.29.3 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2
e.29.4 2 2 2 1 2 2 2 2 2 2 2 2 2 2 2 2 2 2
e.29.5 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2
e.31.1 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2
e.31.2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2
e.31.3 2 2 2 0 2 2 2 1 2 2 2 2 1 2 2 1 2 2
e.32.5 2 0 2 2 2 2 0 2 0 0 2 2 2 0 2 2 2 2
e.33.2 2 0 2 0 0 2 0 2 0 0 2 2 2 0 2 2 2 2
e.35.2 2 0 2 1 2 2 0 2 0 2 2 2 2 0 2 2 2 2
stotal 112 74 112 92 104 108 98 100 92 86 112 106 105 92 108 105 104 112
[C90: Documents on implementation defined behaviors] (13 items)
d.1.1 2 0 2 0 0 2 0 0 0 0 2 2 0 0 2 0 0 2
d.1.2 4 2 4 4 4 4 0 4 0 0 4 4 4 0 4 4 2 4
d.1.3 2 0 2 0 0 2 0 0 2 2 2 2 0 0 2 0 2 2
d.1.4 4 0 4 4 4 4 0 4 4 2 4 4 4 4 4 4 2 4
d.1.5 4 2 4 4 2 4 4 4 4 4 4 2 4 4 2 4 4 4
d.1.6 2 0 2 0 0 1 0 0 0 0 2 1 0 0 1 0 0 2
d.2.1 2 0 2 2 2 2 2 0 0 0 2 2 2 0 2 2 0 2
d.2.2 2 0 2 2 0 2 0 0 0 0 2 2 2 0 2 2 0 2
d.2.3 2 0 2 0 0 0 0 0 0 0 2 0 0 0 0 0 0 2
d.2.4 2 0 2 0 0 0 0 0 0 0 2 0 0 0 0 0 0 2
d.2.5 2 0 2 0 0 0 0 2 0 0 2 0 2 0 0 2 0 2
d.2.6 2 0 2 2 0 0 0 2 0 0 2 0 2 0 0 2 0 2
d.2.7 2 0 2 2 0 2 0 2 0 0 2 2 2 0 2 2 0 2
stotal 32 4 32 20 12 23 6 18 10 8 32 21 22 8 21 22 10 32
[C90: Degree of Standard C conformance] (171 items)
mttl90 598 390 596 530 570 585 550 550 534 476 598 581 561 530 583 561 555 598
[C99: Conformance to new features] (20 items)
n.dslcom 4 0 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4
n.ucn1 8 0 0 0 0 6 8 2 0 2 8 6 8 0 6 8 6 8
n.ucn2 2 0 0 0 0 0 2 0 0 0 2 2 2 0 2 2 0 2
n.ppnum 4 0 4 0 4 4 4 0 0 0 4 4 0 4 4 0 0 4
n.line 2 0 2 2 2 2 2 2 0 2 2 2 2 0 2 2 2 2
n.pragma 6 0 6 0 0 6 6 0 0 2 6 6 0 0 6 0 2 6
n.llong 10 0 0 0 10 10 8 10 0 0 10 10 10 0 10 10 10 10
n.vargs 10 0 10 0 10 10 10 0 0 10 10 10 10 2 10 10 10 10
n.stdmac 4 0 2 0 0 4 4 0 0 4 4 4 0 0 4 0 4 4
n.nularg 6 0 6 0 6 6 6 2 0 6 6 6 2 0 6 2 6 6
n.tlimit 18 0 18 14 18 18 17 18 14 18 18 18 18 12 18 18 16 18
e.ucn 4 0 0 0 0 0 2 0 0 2 4 0 2 0 0 2 2 4
e.intmax 2 0 0 0 2 2 2 0 0 0 2 1 0 0 1 0 2 2
e.pragma 2 0 2 0 0 2 2 0 0 2 2 2 0 0 2 0 2 2
e.vargs1 2 0 0 0 0 2 1 0 0 1 2 2 0 0 2 0 2 2
e.vargs2 2 0 2 0 0 0 0 0 0 0 2 0 0 0 0 0 0 2
d.pragma 2 0 2 0 0 2 2 2 0 0 2 2 2 0 2 2 2 2
d.predef 6 0 0 0 0 6 6 0 0 0 6 6 0 0 6 0 0 6
d.ucn 2 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 2
d.mbiden 2 0 0 0 0 2 2 1 0 0 2 2 1 0 2 1 0 2
mttl99 98 0 58 20 56 86 88 41 18 53 98 87 61 22 87 61 70 98
[C++: Conformance to new features not in C90] (9 items)
n.dslcom 4 0 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4
n.ucn1 4 0 0 0 0 4 4 2 0 2 4 4 2 0 4 2 2 4
n.cnvucn 4 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0
n.bool 2 0 0 0 0 2 0 0 0 2 2 2 0 0 2 0 0 2
n.token1 2 0 2 0 0 2 0 0 2 2 2 2 0 2 2 0 2 2
n.token2 2 0 0 0 0 2 0 2 0 2 2 2 2 0 2 2 2 2
n.cplus 4 0 2 2 2 2 0 4 0 4 4 2 4 0 2 4 4 4
e.operat 2 0 0 0 0 2 0 0 0 2 2 2 0 0 2 0 2 2
d.tlimit 2 0 2 0 0 2 0 1 0 0 2 2 1 0 2 1 0 2
mttl++ 26 0 10 6 6 20 9 13 6 18 22 20 13 6 20 13 16 22
[C90: Qualities / 1 : handling of multibyte character] (1 item)
m.36.2 7 0 2 2 0 0 0 4 0 0 7 5 2 0 5 2 0 7
stotal 7 0 2 2 0 0 0 4 0 0 7 5 2 0 5 2 0 7
[C90: Qualities / 2 : diagnosis of undefined behaviors] (29 items)
u.1.1 1 0 1 0 1 1 0 0 0 1 1 1 0 0 1 0 1 1
u.1.2 1 0 1 0 1 1 0 1 0 1 1 1 0 0 1 0 1 1
u.1.3 1 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
u.1.4 1 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
u.1.5 1 1 1 1 1 1 1 1 1 0 1 1 1 1 1 1 1 1
u.1.6 1 0 1 0 1 1 0 0 1 0 1 1 0 1 1 0 0 1
u.1.7 9 0 1 0 0 0 0 0 0 0 6 6 0 0 6 0 0 9
u.1.8 1 1 1 0 1 1 0 0 0 0 1 0 0 1 0 0 0 1
u.1.9 1 1 1 0 1 1 1 0 1 0 1 0 0 1 0 0 0 1
u.1.10 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
u.1.11 1 1 1 1 0 1 1 1 1 1 1 1 1 1 1 1 1 1
u.1.12 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
u.1.13 1 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
u.1.14 1 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
u.1.15 1 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
u.1.16 1 0 1 1 1 1 1 1 1 0 1 1 1 1 1 1 1 1
u.1.17 2 0 2 0 1 1 0 0 1 0 2 1 0 1 1 0 1 2
u.1.18 1 0 1 0 1 1 1 0 0 0 1 1 0 0 1 0 1 1
u.1.19 2 0 2 0 0 1 1 0 0 1 2 1 0 0 1 0 1 2
u.1.20 1 0 1 1 1 1 1 1 0 1 1 1 1 1 1 1 1 1
u.1.21 2 0 2 1 0 1 2 2 2 2 2 1 2 2 1 2 2 2
u.1.22 1 0 1 0 0 1 1 0 1 1 1 1 0 1 1 0 1 1
u.1.23 1 1 1 0 1 0 0 1 1 0 1 0 1 1 0 1 0 1
u.1.24 2 0 2 0 0 0 0 0 0 0 2 0 0 0 0 0 2 2
u.1.25 1 0 1 0 0 1 0 0 0 0 1 1 0 0 1 0 1 1
u.1.27 1 1 1 1 0 1 1 1 1 0 1 1 1 1 1 1 1 1
u.1.28 1 1 1 1 0 1 1 1 1 0 1 1 1 1 1 1 1 1
u.2.1 1 1 1 1 1 1 0 1 0 1 1 1 1 0 1 1 1 1
u.2.2 1 0 1 1 0 1 1 0 0 0 1 1 0 0 1 0 0 1
stotal 41 10 33 16 19 26 20 18 19 16 38 30 17 21 30 17 25 41
[C90: Qualities / 3 : Diagnosis of unspecified behaviors] (2 items)
s.1.1 2 0 2 0 0 0 2 0 0 0 2 0 0 0 0 0 0 2
s.1.2 2 0 2 0 0 0 0 0 0 0 2 0 0 0 0 0 0 2
stotal 4 0 4 0 0 0 2 0 0 0 4 0 0 0 0 0 0 4
[C90: Qualities / 4 : Diagnosis of suspicious cases] (12 items)
w.1.1 4 4 4 0 4 4 0 0 0 0 4 4 0 0 4 0 0 4
w.1.2 4 0 4 0 0 0 0 0 0 2 4 0 0 0 0 0 2 4
w.2.1 2 0 2 1 0 0 0 0 0 0 2 2 0 0 2 0 0 2
w.2.2 1 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 1
w.3.1 1 1 1 0 0 0 0 0 0 0 1 0 0 1 0 0 0 1
w.3.3 1 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 1
w.3.4 1 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 1
w.3.5 1 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 1
w.3.6 1 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 1
w.3.7 1 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 1
w.3.8 1 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 1
w.3.9 1 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 1
stotal 19 5 19 1 4 4 0 0 0 2 19 6 0 1 6 0 2 19
[C90: Qualities / 5 : Other features] (17 items)
q.1.1 9 0 9 6 9 9 8 7 4 9 9 9 7 5 9 7 8 9
q.1.2 10 6 10 4 8 10 4 4 4 4 10 10 4 4 10 4 4 10
q.1.3 4 4 4 2 4 4 4 4 4 4 4 4 4 4 4 4 4 4
q.1.4 20 10 20 10 20 20 20 10 20 10 20 20 10 20 20 10 10 20
q.2.1 4 2 4 2 4 4 4 4 2 4 4 4 4 2 4 4 4 4
q.2.2 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4
q.2.3 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2
q.2.4 2 2 2 0 2 2 0 0 0 0 2 2 0 0 2 0 0 2
q.2.5 2 0 2 0 0 0 0 0 0 0 2 0 0 0 0 0 0 2
q.2.6 4 0 4 2 4 4 2 4 2 0 4 4 4 2 4 4 0 4
q.2.7 10 4 6 4 8 8 4 4 4 2 8 8 4 4 8 4 4 8
q.2.8 10 0 6 2 2 2 0 6 4 4 8 2 6 4 2 6 4 8
q.2.9 6 2 2 0 2 2 0 0 0 2 2 2 0 0 2 0 2 4
q.3.1 20 10 8 8 14 12 8 10 10 6 8 12 10 10 12 10 6 8
q.3.2 20 20 20 18 16 16 18 16 18 14 18 14 14 18 12 14 12 16
q.3.3 20 10 14 0 10 12 8 0 0 8 14 12 0 0 12 0 10 16
q.4.1 10 2 6 6 4 6 2 4 6 4 4 6 4 6 6 4 4 8
stotal 157 78 123 70 113 117 88 79 84 77 123 115 77 85 113 77 78 129
[C90: Qualities] (61 items)
mttl90 228 93 181 89 136 147 110 101 103 95 191 156 96 107 154 96 105 200
[C99: Qualities of new features] (3 items)
u.line 2 0 2 0 1 1 0 0 0 0 2 1 0 0 1 0 2 2
u.concat 1 0 1 0 0 1 0 0 0 0 1 1 0 0 1 0 0 1
w.tlimit 8 0 8 0 0 0 3 2 0 0 8 0 2 0 0 2 0 8
mttl99 11 0 11 0 1 2 3 2 0 0 11 2 2 0 2 2 2 11
[C++: Qualities of features not in C90] (1 item)
u.cplus 1 0 1 1 0 0 0 1 0 1 1 0 1 0 0 1 1 1
mttl++ 1 0 1 1 0 0 0 1 0 1 1 0 1 0 0 1 1 1
[Overall] (265 items)
gtotal 962 483 857 646 769 840 760 708 661 643 921 846 734 665 846 734 749 930
D M B G G u V L W M G V L G V W M
E C C C C c C C a C C C C C C A C
C P C C C p 2 C v P C 2 C C 2 V P
U P 5 2 3 p 0 0 e P 3 0 0 4 0 E P
S 2 5 9 2 1 0 3 1 2 4 0 6 1 0 2 2
C 0 C 5 3 3 0 0 4 3 5 0 1 8 0 7
P P 3 8 3 2
P P
max 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17
</pre>
<br>
<h2><a name="6.3" href="#toc.6.3">6.3. Characteristics of Each Preprocessor</a></h2>
<p>1 : Linux / / DECUS cpp</p>
<blockquote>
<p>This was written around the early stage of ANSI draft and the Standard conformance level is low by now. Diagnostic messages are adequate, however, there is almost no documentation. It is a well-structured stable program.</p>
<p>The portability of source is high and it has been ported to some compiler systems. The source code is easy to read as if reading a textbook and you can learn a lot from by just reading it. I modeled <b>mcpp</b> after this source.</p>
</blockquote>
<p>3 : WIN32 / Borland C++ V.5.5J / cpp32</p>
<blockquote>
<p>The C90 conformance level is relatively high and troublesome shift-JIS is respectably supported. Documents are well provided. Although e_* usually issues diagnostic messages, most of them are hasty and the quality is not good.</p>
<p>"Quality other than Standards" is poor and there are no special extension features. Not many diagnostic messages are issued for undefined parts and the program runs away sometimes. Supporting Standards only seems to be the best it could do.</p>
<p>Different from Turbo C, the speed is no longer fast. Merits in one-pass compiler seem to have disappeared and only demerits seem to have remained. I wonder how long Borland continue this style.</p>
</blockquote>
<p>4 : Linux, CygWIN / GCC V.2.95.3 / cpp0</p>
<blockquote>
<p>The C90 and C95 Standard conformance level is quite high and diagnostic messages are accurate. The behavior is near stable and the speed is extremely fast. There are plentiful options which are almost too abundant. <b>mcpp</b> also modeled after some of those options.</p>
<p>Though there were a few painful bugs in older versions, V.2.95 has almost no bugs.</p>
<p>The remaining issues are that new specifications in C99 and C++98 have not been implemented, there are not enough diagnostic messages, documentation is lacking, there are many extension features against Standards which do not use #pragma, that many obsolete pre-Standard specifications are hidden, and that multi-byte character encoding support is half-finished and does not reach practical level.</p>
<p>The cpp.info document is excellent as the explanation of overall GCC/cpp and Standard C preprocessing. However, it is really too bad that documentation for implementation-defined areas do not exist in CygWIN, FreeBSD, or Linux. "Portability" is not only for programs.</p>
<p>The source is a full of patches and difficult to read, and the program structure is still dragging old macro processor structure. However, since overall GCC compiler systems are good, this is ported to many systems.</p>
</blockquote>
<p>5 : Linux / GCC V.3.2 / cpp0</p>
<blockquote>
<p>GCC V.3 changed the source for preprocessing completely from V.2. At the same time, it changed documentations completely. Token-based principles are fully enforced, warnings are issued while allowing pre-Standard specifications, and the number of undocumented specifications decreased. On the whole, it has improved to the direction I had hoped for to a large extent. I suspect that future improvements will be easier since the program structure has changed completely.</p>
<p>Diagnostic messages, documentations, C99 support, multi-byte character support are not enough yet. The speed is slightly slower than V.2, but it is still one of the faster ones.</p>
<p>However, it is troublesome that header files became complex and setting the search order of include directories is getting complicated. Also, while old options are no longer necessary, many new options are introduced and it is taking forever for options to get organized. It is unfortunate that the internal interface between preprocessing and compilation parts is complicated for some reasons although preprocessing become combined with the compiler proper in V.3.</p>
</blockquote>
<p>6 : Linux / / ucpp (V.1.3)</p>
<blockquote>
<p>The characteristics are the support for C99, open source, and portable. The Standard conformance level is rather high. This version is supposed to support UCN and UTF-8, but the support is insufficient. The diagnostic messages are somewhat poor. Documentation is not sufficient, either.</p>
</blockquote>
<p>7 : WIN32 / Visual C++ 2003 / cl<br>
12 : WIN32 / Visual C++ 2005 / cl<br>
15 : WIN32 / Visual C++ 2008 / cl</p>
<blockquote>
<p>Though in 2003 few C99 specifications are implemented, more than half of them are implemented in 2005. There remain, however, some bugs regarding the specifications for C90 and prior. The most fundamental problem is confusion in the translation phases. The upgrades must have been rework of some very old source code.</p>
<p>The diagnostic messages are often somewhat off the point. An error often terminates the preprocessing which makes this software bothersome to use. Updating of the manuals is sometimes far behind the implementation.</p>
<p>The merits are large translation limits and a relatively large number of #pragma. #pragma setlocale in particular is useful. However, it is problematic that the #pragma line is macro-expanded even in C90 but that #pragma sub-directive uses user's name space.</p>
<p>2008 is almost the same with 2005 as for preprocessing.
The only difference is implementation of __pragma() operator which is a substitute of _Pragma() of C99.
The system headers used by Visual C++ have had only a few problems on the whole up to 2005.
On 2008, however, number of macros with '$' suddenly increased for some reason.
</blockquote>
<p>8 : WIN32 / LCC-Win32 2003-08 / lcc<br>
13 : WIN32 / LCC-Win32 2006-03 / lcc</p>
<blockquote>
<p>Jacob Navia modified the preprocessing part of the source code for Plan9 by Dennis Ritchie, but this lacks in debugging and there are quite a number of bugs in the #if expression evaluation and others. The specifications since C95 are not supported. Lack of documentation.</p>
<p>There are few differences of preprocessing between 2003-08 and 2006-03.</p>
</blockquote>
<p>9 : WIN32, Linux, etc. / / wave (V.1.0)</br>
16: WIN32, Linux, etc. / / wave (V.2.0)</p>
<blockquote>
<p>This preprocessor has been developed for "meta-programming" of C++ STL as a primary purpose. Wave has an unique construction: it is consisted of mainly C++ libraries, and its source is consisted of mainly header files. The examples of meta-programming use recursive macros heavily, and Wave expands those macros as GCC / cpp or -@compat option of <b>mcpp</b>, i.e. limiting the scope of "inhibition of once-replaced macro's re-replacement" narrower than the Standard's wording. (see <a href=#3.4.26>3.4.26</a>.)</p>
<p>Wave intends also to be used as a usual preprocessor, and intends conforming to C++98 and C99. Though the degree of perfection was not high in V.1.0, it was greatly improved in V.2.0. It was reported that a lot of bugs were fixed after V.1.0 using the validation suite of <b>mcpp</b>. Further improvement of diagnostics and documents are still desired. Another problem is that a few mistakes are found in the author's interpretation of the Standards expressed in wave's diagnostics and its accompanying testcases.</p>
</blockquote>
<p>11 : Linux / GCC V.3.4.3 / cc1, cc1plus<br>
14 : Linux / GCC V.4.1.1 / cc1, cc1plus</p>
<blockquote>
<p>The scoring is almost the same with V.3.2. However, the construction of preprocessing has changed largely. Although V.3.2 seemed to proceed in the direction to portability, GCC has changed the direction on V.3.3 and 3.4. It has become one huge and complex compiler, removing independent preprocessor, predefining many macros and restoring some old specifications which was once obsoleted by V.3.2. It is a question whether these changes can be said improvements. It has also given a privileged place to UTF-8 among many encodings of multi-byte character. I am afraid that it might narrow the wide variety of multi-lingualization.</p>
<p>V.4.1 has not big differences from V.3.4 as for preprocessing.</p>
</blockquote>
<p>2 : FreeBSD, DJGPP, WIN32, MS-DOS, OS-9/09 / / <b>mcpp</b> (V.2.0)<br>
10 : FreeBSD, Linux, CygWIN, WIN32, MS-DOS / / <b>mcpp</b> (V.2.4)<br>
17 : FreeBSD, Linux, Mac OS X, CygWIN, MinGW, WIN32 / / <b>mcpp</b> (V.2.7.2)</p>
<blockquote>
<p>Since I created and tested this myself, the conformance level is the best, of course. It should be the world's most accurate preprocessor. The plentifulness and accuracy of diagnostic messages and the detailed documentation is also the best. Useful options and #pragma directives are provided. The C99 specification is fully supported in V.2.3 and later.
The portability of source is also the best.</p>
However, there are some features still to be implemented.
So I would appreciate your contribution.</p>
</blockquote>
<br>
<h2><a name="6.4" href="#toc.6.4">6.4. Overall Review</a></h2>
<p>As we test many preprocessors, we can find that nowadays many have high level of C90 Standard conformance. However, each compiler system still has many issues. I am not going to speak for <b>mcpp</b> since most of the items score full.</p>
<p>More compiler systems can process the n_* samples correctly now. GCC 2.95 and later, BC (Borland C) 5.5, LCC-Win32 2003-08 and later, Visual C++ (VC) 2003 and later, Ucpp 1.3, and Wave 2.0 have reached the level with not so many problems in practice. However, each compiler system has unexpected bugs.</p>
<p>The most surprising is that compiler systems including Visual C often have the division by 0 errors in n_13_7.t (n_13_7.c.) The basic specification of C, the "short-circuit evaluation" by &amp;&amp;, || and a ternary operator, is not handled. Borland C issues a warning in n_13_7.c and it issues only the same warning for the real division by 0 in e_14_9.c as well. In Turbo C, the real division by 0 and partial expression with skipped evaluation caused the same error while the same diagnostic message is downgraded only to warning in Borland C. This is an example of a "hasty diagnostic message" in this compiler system.</p>
<p>In the C90 specification, there are some with errors in the stringizing implementation using the # operator.</p>
<p>The specifications added by Amendment 1, Corrigendum 1 are implemented to some extent by GCC 2.95 and later, VC 2003 and later, and Ucpp.</p>
<p>In the C99 specification, only GCC 3.2 and later and Ucpp implements largely but not completely. The // comments has been implemented by many compiler systems for quite some time. In addition, GCC has long long, has considerable room in translation limits, and properly processes empty arguments of macros. GCC has variable argument macro of its own specification, but the one in the C99 specification is also implemented since 2.95. _Pragma() is implemented since 3.2. UCN is implemented by Ucpp and VC 2005 and later only. GCC 3.2 and later implements UCN in string literals and character constants only. Wave 2.0 implements more than half of C99 specifications.</p>
<p>In C++98, GCC 3.2 and later, Wave and VC 2005 and later implements most of the specifications.</p>
<p>The queer specification of C++98 to convert extended-characters to UCN is not yet implemented by any preprocessor.</p>
<p>In processing the implementation-defined area in i_*, many cannot handle wide characters in the #if expression. Though this is specified in Standards, using not only wide characters, but also character constants in the #if expression is almost meaningless and it will not hurt even if these cannot be used. This type of meaningless specification should be removed from the Standard.</p>
<p>Visual C supports relatively many encodings for multi-byte characters, though not enough. Other preprocessors are poor. The implementation of GCC 2.95,3.2 are half-finished and does not reach a practical level. GCC 3.4-4.1 has begun to support many encodings, by converting all encodings to UTF-8. The actual implementation is, however, not yet practical level on some encodings.</p>
<p>In the systems using shift-JIS or BIG-5, tokenization of literals and stringizing using the # operator requires attention. Visual C supports these well. Also BC 5.5J support shift-JIS.</p>
<p>In the diagnostic messages for e_*, GCC 2.95 and later are superior. Though Visual C and Ucpp issue diagnostics to comparatively many items, they are often vague or off-target. Very few preprocessors issue diagnostic messages to the overflow in the #if expression and only BC, Ucpp and GCC do to some extent.</p>
<p>In documents for implementation-defined areas, GCC 3.2 and later are at adequate level and the rest are all very poor.</p>
<p>In the diagnostics for u_*, GCC 3.2 and later is only adequate. The rest are very poor. I don't think it is acceptable for compiler systems not to do anything just because the result is undefined.</p>
<p>Almost no compiler systems handle s_* and w_*. It is unexpected that very few compiler systems issue even a warning for nested comments.</p>
<p>In "other various quality", GCC stand out with plentiful options, accurate diagnostic messages, high speed, and portability.</p>
<p>Overall, GCC V.3.2 and later, excels the most in the Standard conformance level, ease of use, and stability without many big problems.</p>
<p>Certainly, it is understood that <b>mcpp</b> exceeds in most aspects though only speed is inferior.</p>
<p>After the huge volume of testing, what I realize is the importance of test samples. <b>mcpp</b> is the result of creating samples and debugging in parallel. Since you cannot notice the existence of bugs without enough samples, it is anything but debugging.</p>
<p>If Standards come with this type of exhaustive test samples, the quality of each compiler system will be exponentially improved. Also, creating exhaustive test samples reveals the problems in Standards at the same time. Test samples are the illustration of Standards.</p>
<br>
<h2><a name="6.5" href="#toc.6.5">6.5. Test Reports and Comments</a></h2>
<p>I look forward to opinions about this Validation Suite and preprocessing test reports for various compiler systems using this tool. Please use the "Open Discussion Forum" at:</p>
<p><a href="http://mcpp.sourceforge.net/">http://mcpp.sourceforge.net/</a></p>
<p>or email.</p>
<p>If you perform the detail testing of a preprocessor, cut out the <a href="#6.2">6.2</a> score table and send it. To calculate the total score, please compile and use tool/total.c. The score table is cpp_test.old and if</p>
<p>total 18 cpp_test.old cpp_test.new</p>
<p>each field of stotal (sub-total), mtotal (mid-total), and gtotal (grand-total) is written and output to cpp_test.new. Specify the compiler system number at "18".</p>
<p>You can test automatically GCC by the testsuite edition of Validation Suite. I am waiting for the test reports on various versions of GCC. Please send me the log files (gcc.sum and gcc.log), and I will supplement my testsuite edition with the diagnostics of various versions if any differences exist.</p>
<p>Also, the development of Validation Suite and <b>mcpp</b> are in progress in the mcpp project above in SourceForge. Please send me email if you would like to join the development.</p>
</body>
</html>
Reference in New Issue View Git Blame Copy Permalink