|
- <?xml version="1.0" encoding="UTF-8" standalone="no"?>
- <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"><html xmlns="http://www.w3.org/1999/xhtml"><head><meta http-equiv="Content-Type" content="text/html; charset=UTF-8" /><title>32.3. Variant Comparison Files</title><link rel="stylesheet" type="text/css" href="stylesheet.css" /><link rev="made" href="pgsql-docs@lists.postgresql.org" /><meta name="generator" content="DocBook XSL Stylesheets V1.79.1" /><link rel="prev" href="regress-evaluation.html" title="32.2. Test Evaluation" /><link rel="next" href="regress-tap.html" title="32.4. TAP Tests" /></head><body><div xmlns="http://www.w3.org/TR/xhtml1/transitional" class="navheader"><table width="100%" summary="Navigation header"><tr><th colspan="5" align="center">32.3. Variant Comparison Files</th></tr><tr><td width="10%" align="left"><a accesskey="p" href="regress-evaluation.html" title="32.2. Test Evaluation">Prev</a> </td><td width="10%" align="left"><a accesskey="u" href="regress.html" title="Chapter 32. Regression Tests">Up</a></td><th width="60%" align="center">Chapter 32. Regression Tests</th><td width="10%" align="right"><a accesskey="h" href="index.html" title="PostgreSQL 12.4 Documentation">Home</a></td><td width="10%" align="right"> <a accesskey="n" href="regress-tap.html" title="32.4. TAP Tests">Next</a></td></tr></table><hr></hr></div><div class="sect1" id="REGRESS-VARIANT"><div class="titlepage"><div><div><h2 class="title" style="clear: both">32.3. Variant Comparison Files</h2></div></div></div><p>
- Since some of the tests inherently produce environment-dependent
- results, we have provided ways to specify alternate <span class="quote">“<span class="quote">expected</span>”</span>
- result files. Each regression test can have several comparison files
- showing possible results on different platforms. There are two
- independent mechanisms for determining which comparison file is used
- for each test.
- </p><p>
- The first mechanism allows comparison files to be selected for
- specific platforms. There is a mapping file,
- <code class="filename">src/test/regress/resultmap</code>, that defines
- which comparison file to use for each platform.
- To eliminate bogus test <span class="quote">“<span class="quote">failures</span>”</span> for a particular platform,
- you first choose or make a variant result file, and then add a line to the
- <code class="filename">resultmap</code> file.
- </p><p>
- Each line in the mapping file is of the form
- </p><pre class="synopsis">
- testname:output:platformpattern=comparisonfilename
- </pre><p>
- The test name is just the name of the particular regression test
- module. The output value indicates which output file to check. For the
- standard regression tests, this is always <code class="literal">out</code>. The
- value corresponds to the file extension of the output file.
- The platform pattern is a pattern in the style of the Unix
- tool <code class="command">expr</code> (that is, a regular expression with an implicit
- <code class="literal">^</code> anchor at the start). It is matched against the
- platform name as printed by <code class="command">config.guess</code>.
- The comparison file name is the base name of the substitute result
- comparison file.
- </p><p>
- For example: some systems lack a working <code class="literal">strtof</code> function,
- for which our workaround causes rounding errors in the
- <code class="filename">float4</code> regression test.
- Therefore, we provide a variant comparison file,
- <code class="filename">float4-misrounded-input.out</code>, which includes
- the results to be expected on these systems. To silence the bogus
- <span class="quote">“<span class="quote">failure</span>”</span> message on <span class="systemitem">HP-UX 10</span>
- platforms, <code class="filename">resultmap</code> includes:
- </p><pre class="programlisting">
- float4:out:hppa.*-hp-hpux10.*=float4-misrounded-input.out
- </pre><p>
- which will trigger on any machine where the output of
- <code class="command">config.guess</code> matches <code class="literal">hppa.*-hp-hpux10.*</code>.
- Other lines in <code class="filename">resultmap</code> select the variant comparison
- file for other platforms where it's appropriate.
- </p><p>
- The second selection mechanism for variant comparison files is
- much more automatic: it simply uses the <span class="quote">“<span class="quote">best match</span>”</span> among
- several supplied comparison files. The regression test driver
- script considers both the standard comparison file for a test,
- <code class="literal"><em class="replaceable"><code>testname</code></em>.out</code>, and variant files named
- <code class="literal"><em class="replaceable"><code>testname</code></em>_<em class="replaceable"><code>digit</code></em>.out</code>
- (where the <em class="replaceable"><code>digit</code></em> is any single digit
- <code class="literal">0</code>-<code class="literal">9</code>). If any such file is an exact match,
- the test is considered to pass; otherwise, the one that generates
- the shortest diff is used to create the failure report. (If
- <code class="filename">resultmap</code> includes an entry for the particular
- test, then the base <em class="replaceable"><code>testname</code></em> is the substitute
- name given in <code class="filename">resultmap</code>.)
- </p><p>
- For example, for the <code class="literal">char</code> test, the comparison file
- <code class="filename">char.out</code> contains results that are expected
- in the <code class="literal">C</code> and <code class="literal">POSIX</code> locales, while
- the file <code class="filename">char_1.out</code> contains results sorted as
- they appear in many other locales.
- </p><p>
- The best-match mechanism was devised to cope with locale-dependent
- results, but it can be used in any situation where the test results
- cannot be predicted easily from the platform name alone. A limitation of
- this mechanism is that the test driver cannot tell which variant is
- actually <span class="quote">“<span class="quote">correct</span>”</span> for the current environment; it will just pick
- the variant that seems to work best. Therefore it is safest to use this
- mechanism only for variant results that you are willing to consider
- equally valid in all contexts.
- </p></div><div class="navfooter"><hr /><table width="100%" summary="Navigation footer"><tr><td width="40%" align="left"><a accesskey="p" href="regress-evaluation.html">Prev</a> </td><td width="20%" align="center"><a accesskey="u" href="regress.html">Up</a></td><td width="40%" align="right"> <a accesskey="n" href="regress-tap.html">Next</a></td></tr><tr><td width="40%" align="left" valign="top">32.2. Test Evaluation </td><td width="20%" align="center"><a accesskey="h" href="index.html">Home</a></td><td width="40%" align="right" valign="top"> 32.4. TAP Tests</td></tr></table></div></body></html>
|