|
- <!DOCTYPE html>
-
- <html lang="en" data-content_root="../">
- <head>
- <meta charset="utf-8" />
- <meta name="viewport" content="width=device-width, initial-scale=1.0" /><meta name="viewport" content="width=device-width, initial-scale=1" />
- <meta property="og:title" content="heapq — Heap queue algorithm" />
- <meta property="og:type" content="website" />
- <meta property="og:url" content="https://docs.python.org/3/library/heapq.html" />
- <meta property="og:site_name" content="Python documentation" />
- <meta property="og:description" content="Source code: Lib/heapq.py This module provides an implementation of the heap queue algorithm, also known as the priority queue algorithm. Heaps are binary trees for which every parent node has a va..." />
- <meta property="og:image" content="https://docs.python.org/3/_static/og-image.png" />
- <meta property="og:image:alt" content="Python documentation" />
- <meta name="description" content="Source code: Lib/heapq.py This module provides an implementation of the heap queue algorithm, also known as the priority queue algorithm. Heaps are binary trees for which every parent node has a va..." />
- <meta property="og:image:width" content="200" />
- <meta property="og:image:height" content="200" />
- <meta name="theme-color" content="#3776ab" />
-
- <title>heapq — Heap queue algorithm — Python 3.12.3 documentation</title><meta name="viewport" content="width=device-width, initial-scale=1.0">
-
- <link rel="stylesheet" type="text/css" href="../_static/pygments.css?v=80d5e7a1" />
- <link rel="stylesheet" type="text/css" href="../_static/pydoctheme.css?v=bb723527" />
- <link id="pygments_dark_css" media="(prefers-color-scheme: dark)" rel="stylesheet" type="text/css" href="../_static/pygments_dark.css?v=b20cc3f5" />
-
- <script src="../_static/documentation_options.js?v=2c828074"></script>
- <script src="../_static/doctools.js?v=888ff710"></script>
- <script src="../_static/sphinx_highlight.js?v=dc90522c"></script>
-
- <script src="../_static/sidebar.js"></script>
-
- <link rel="search" type="application/opensearchdescription+xml"
- title="Search within Python 3.12.3 documentation"
- href="../_static/opensearch.xml"/>
- <link rel="author" title="About these documents" href="../about.html" />
- <link rel="index" title="Index" href="../genindex.html" />
- <link rel="search" title="Search" href="../search.html" />
- <link rel="copyright" title="Copyright" href="../copyright.html" />
- <link rel="next" title="bisect — Array bisection algorithm" href="bisect.html" />
- <link rel="prev" title="collections.abc — Abstract Base Classes for Containers" href="collections.abc.html" />
- <link rel="canonical" href="https://docs.python.org/3/library/heapq.html" />
-
-
-
-
-
- <style>
- @media only screen {
- table.full-width-table {
- width: 100%;
- }
- }
- </style>
- <link rel="stylesheet" href="../_static/pydoctheme_dark.css" media="(prefers-color-scheme: dark)" id="pydoctheme_dark_css">
- <link rel="shortcut icon" type="image/png" href="../_static/py.svg" />
- <script type="text/javascript" src="../_static/copybutton.js"></script>
- <script type="text/javascript" src="../_static/menu.js"></script>
- <script type="text/javascript" src="../_static/search-focus.js"></script>
- <script type="text/javascript" src="../_static/themetoggle.js"></script>
-
- </head>
- <body>
- <div class="mobile-nav">
- <input type="checkbox" id="menuToggler" class="toggler__input" aria-controls="navigation"
- aria-pressed="false" aria-expanded="false" role="button" aria-label="Menu" />
- <nav class="nav-content" role="navigation">
- <label for="menuToggler" class="toggler__label">
- <span></span>
- </label>
- <span class="nav-items-wrapper">
- <a href="https://www.python.org/" class="nav-logo">
- <img src="../_static/py.svg" alt="Python logo"/>
- </a>
- <span class="version_switcher_placeholder"></span>
- <form role="search" class="search" action="../search.html" method="get">
- <svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" viewBox="0 0 24 24" class="search-icon">
- <path fill-rule="nonzero" fill="currentColor" d="M15.5 14h-.79l-.28-.27a6.5 6.5 0 001.48-5.34c-.47-2.78-2.79-5-5.59-5.34a6.505 6.505 0 00-7.27 7.27c.34 2.8 2.56 5.12 5.34 5.59a6.5 6.5 0 005.34-1.48l.27.28v.79l4.25 4.25c.41.41 1.08.41 1.49 0 .41-.41.41-1.08 0-1.49L15.5 14zm-6 0C7.01 14 5 11.99 5 9.5S7.01 5 9.5 5 14 7.01 14 9.5 11.99 14 9.5 14z"></path>
- </svg>
- <input placeholder="Quick search" aria-label="Quick search" type="search" name="q" />
- <input type="submit" value="Go"/>
- </form>
- </span>
- </nav>
- <div class="menu-wrapper">
- <nav class="menu" role="navigation" aria-label="main navigation">
- <div class="language_switcher_placeholder"></div>
-
- <label class="theme-selector-label">
- Theme
- <select class="theme-selector" oninput="activateTheme(this.value)">
- <option value="auto" selected>Auto</option>
- <option value="light">Light</option>
- <option value="dark">Dark</option>
- </select>
- </label>
- <div>
- <h3><a href="../contents.html">Table of Contents</a></h3>
- <ul>
- <li><a class="reference internal" href="#"><code class="xref py py-mod docutils literal notranslate"><span class="pre">heapq</span></code> — Heap queue algorithm</a><ul>
- <li><a class="reference internal" href="#basic-examples">Basic Examples</a></li>
- <li><a class="reference internal" href="#priority-queue-implementation-notes">Priority Queue Implementation Notes</a></li>
- <li><a class="reference internal" href="#theory">Theory</a></li>
- </ul>
- </li>
- </ul>
-
- </div>
- <div>
- <h4>Previous topic</h4>
- <p class="topless"><a href="collections.abc.html"
- title="previous chapter"><code class="xref py py-mod docutils literal notranslate"><span class="pre">collections.abc</span></code> — Abstract Base Classes for Containers</a></p>
- </div>
- <div>
- <h4>Next topic</h4>
- <p class="topless"><a href="bisect.html"
- title="next chapter"><code class="xref py py-mod docutils literal notranslate"><span class="pre">bisect</span></code> — Array bisection algorithm</a></p>
- </div>
- <div role="note" aria-label="source link">
- <h3>This Page</h3>
- <ul class="this-page-menu">
- <li><a href="../bugs.html">Report a Bug</a></li>
- <li>
- <a href="https://github.com/python/cpython/blob/main/Doc/library/heapq.rst"
- rel="nofollow">Show Source
- </a>
- </li>
- </ul>
- </div>
- </nav>
- </div>
- </div>
-
-
- <div class="related" role="navigation" aria-label="related navigation">
- <h3>Navigation</h3>
- <ul>
- <li class="right" style="margin-right: 10px">
- <a href="../genindex.html" title="General Index"
- accesskey="I">index</a></li>
- <li class="right" >
- <a href="../py-modindex.html" title="Python Module Index"
- >modules</a> |</li>
- <li class="right" >
- <a href="bisect.html" title="bisect — Array bisection algorithm"
- accesskey="N">next</a> |</li>
- <li class="right" >
- <a href="collections.abc.html" title="collections.abc — Abstract Base Classes for Containers"
- accesskey="P">previous</a> |</li>
-
- <li><img src="../_static/py.svg" alt="Python logo" style="vertical-align: middle; margin-top: -1px"/></li>
- <li><a href="https://www.python.org/">Python</a> »</li>
- <li class="switchers">
- <div class="language_switcher_placeholder"></div>
- <div class="version_switcher_placeholder"></div>
- </li>
- <li>
-
- </li>
- <li id="cpython-language-and-version">
- <a href="../index.html">3.12.3 Documentation</a> »
- </li>
-
- <li class="nav-item nav-item-1"><a href="index.html" >The Python Standard Library</a> »</li>
- <li class="nav-item nav-item-2"><a href="datatypes.html" accesskey="U">Data Types</a> »</li>
- <li class="nav-item nav-item-this"><a href=""><code class="xref py py-mod docutils literal notranslate"><span class="pre">heapq</span></code> — Heap queue algorithm</a></li>
- <li class="right">
-
-
- <div class="inline-search" role="search">
- <form class="inline-search" action="../search.html" method="get">
- <input placeholder="Quick search" aria-label="Quick search" type="search" name="q" id="search-box" />
- <input type="submit" value="Go" />
- </form>
- </div>
- |
- </li>
- <li class="right">
- <label class="theme-selector-label">
- Theme
- <select class="theme-selector" oninput="activateTheme(this.value)">
- <option value="auto" selected>Auto</option>
- <option value="light">Light</option>
- <option value="dark">Dark</option>
- </select>
- </label> |</li>
-
- </ul>
- </div>
-
- <div class="document">
- <div class="documentwrapper">
- <div class="bodywrapper">
- <div class="body" role="main">
-
- <section id="module-heapq">
- <span id="heapq-heap-queue-algorithm"></span><h1><a class="reference internal" href="#module-heapq" title="heapq: Heap queue algorithm (a.k.a. priority queue)."><code class="xref py py-mod docutils literal notranslate"><span class="pre">heapq</span></code></a> — Heap queue algorithm<a class="headerlink" href="#module-heapq" title="Link to this heading">¶</a></h1>
- <p><strong>Source code:</strong> <a class="reference external" href="https://github.com/python/cpython/tree/3.12/Lib/heapq.py">Lib/heapq.py</a></p>
- <hr class="docutils" />
- <p>This module provides an implementation of the heap queue algorithm, also known
- as the priority queue algorithm.</p>
- <p>Heaps are binary trees for which every parent node has a value less than or
- equal to any of its children. This implementation uses arrays for which
- <code class="docutils literal notranslate"><span class="pre">heap[k]</span> <span class="pre"><=</span> <span class="pre">heap[2*k+1]</span></code> and <code class="docutils literal notranslate"><span class="pre">heap[k]</span> <span class="pre"><=</span> <span class="pre">heap[2*k+2]</span></code> for all <em>k</em>, counting
- elements from zero. For the sake of comparison, non-existing elements are
- considered to be infinite. The interesting property of a heap is that its
- smallest element is always the root, <code class="docutils literal notranslate"><span class="pre">heap[0]</span></code>.</p>
- <p>The API below differs from textbook heap algorithms in two aspects: (a) We use
- zero-based indexing. This makes the relationship between the index for a node
- and the indexes for its children slightly less obvious, but is more suitable
- since Python uses zero-based indexing. (b) Our pop method returns the smallest
- item, not the largest (called a “min heap” in textbooks; a “max heap” is more
- common in texts because of its suitability for in-place sorting).</p>
- <p>These two make it possible to view the heap as a regular Python list without
- surprises: <code class="docutils literal notranslate"><span class="pre">heap[0]</span></code> is the smallest item, and <code class="docutils literal notranslate"><span class="pre">heap.sort()</span></code> maintains the
- heap invariant!</p>
- <p>To create a heap, use a list initialized to <code class="docutils literal notranslate"><span class="pre">[]</span></code>, or you can transform a
- populated list into a heap via function <a class="reference internal" href="#heapq.heapify" title="heapq.heapify"><code class="xref py py-func docutils literal notranslate"><span class="pre">heapify()</span></code></a>.</p>
- <p>The following functions are provided:</p>
- <dl class="py function">
- <dt class="sig sig-object py" id="heapq.heappush">
- <span class="sig-prename descclassname"><span class="pre">heapq.</span></span><span class="sig-name descname"><span class="pre">heappush</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">heap</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">item</span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#heapq.heappush" title="Link to this definition">¶</a></dt>
- <dd><p>Push the value <em>item</em> onto the <em>heap</em>, maintaining the heap invariant.</p>
- </dd></dl>
-
- <dl class="py function">
- <dt class="sig sig-object py" id="heapq.heappop">
- <span class="sig-prename descclassname"><span class="pre">heapq.</span></span><span class="sig-name descname"><span class="pre">heappop</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">heap</span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#heapq.heappop" title="Link to this definition">¶</a></dt>
- <dd><p>Pop and return the smallest item from the <em>heap</em>, maintaining the heap
- invariant. If the heap is empty, <a class="reference internal" href="exceptions.html#IndexError" title="IndexError"><code class="xref py py-exc docutils literal notranslate"><span class="pre">IndexError</span></code></a> is raised. To access the
- smallest item without popping it, use <code class="docutils literal notranslate"><span class="pre">heap[0]</span></code>.</p>
- </dd></dl>
-
- <dl class="py function">
- <dt class="sig sig-object py" id="heapq.heappushpop">
- <span class="sig-prename descclassname"><span class="pre">heapq.</span></span><span class="sig-name descname"><span class="pre">heappushpop</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">heap</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">item</span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#heapq.heappushpop" title="Link to this definition">¶</a></dt>
- <dd><p>Push <em>item</em> on the heap, then pop and return the smallest item from the
- <em>heap</em>. The combined action runs more efficiently than <a class="reference internal" href="#heapq.heappush" title="heapq.heappush"><code class="xref py py-func docutils literal notranslate"><span class="pre">heappush()</span></code></a>
- followed by a separate call to <a class="reference internal" href="#heapq.heappop" title="heapq.heappop"><code class="xref py py-func docutils literal notranslate"><span class="pre">heappop()</span></code></a>.</p>
- </dd></dl>
-
- <dl class="py function">
- <dt class="sig sig-object py" id="heapq.heapify">
- <span class="sig-prename descclassname"><span class="pre">heapq.</span></span><span class="sig-name descname"><span class="pre">heapify</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">x</span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#heapq.heapify" title="Link to this definition">¶</a></dt>
- <dd><p>Transform list <em>x</em> into a heap, in-place, in linear time.</p>
- </dd></dl>
-
- <dl class="py function">
- <dt class="sig sig-object py" id="heapq.heapreplace">
- <span class="sig-prename descclassname"><span class="pre">heapq.</span></span><span class="sig-name descname"><span class="pre">heapreplace</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">heap</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">item</span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#heapq.heapreplace" title="Link to this definition">¶</a></dt>
- <dd><p>Pop and return the smallest item from the <em>heap</em>, and also push the new <em>item</em>.
- The heap size doesn’t change. If the heap is empty, <a class="reference internal" href="exceptions.html#IndexError" title="IndexError"><code class="xref py py-exc docutils literal notranslate"><span class="pre">IndexError</span></code></a> is raised.</p>
- <p>This one step operation is more efficient than a <a class="reference internal" href="#heapq.heappop" title="heapq.heappop"><code class="xref py py-func docutils literal notranslate"><span class="pre">heappop()</span></code></a> followed by
- <a class="reference internal" href="#heapq.heappush" title="heapq.heappush"><code class="xref py py-func docutils literal notranslate"><span class="pre">heappush()</span></code></a> and can be more appropriate when using a fixed-size heap.
- The pop/push combination always returns an element from the heap and replaces
- it with <em>item</em>.</p>
- <p>The value returned may be larger than the <em>item</em> added. If that isn’t
- desired, consider using <a class="reference internal" href="#heapq.heappushpop" title="heapq.heappushpop"><code class="xref py py-func docutils literal notranslate"><span class="pre">heappushpop()</span></code></a> instead. Its push/pop
- combination returns the smaller of the two values, leaving the larger value
- on the heap.</p>
- </dd></dl>
-
- <p>The module also offers three general purpose functions based on heaps.</p>
- <dl class="py function">
- <dt class="sig sig-object py" id="heapq.merge">
- <span class="sig-prename descclassname"><span class="pre">heapq.</span></span><span class="sig-name descname"><span class="pre">merge</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="o"><span class="pre">*</span></span><span class="n"><span class="pre">iterables</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">key</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">None</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">reverse</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">False</span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#heapq.merge" title="Link to this definition">¶</a></dt>
- <dd><p>Merge multiple sorted inputs into a single sorted output (for example, merge
- timestamped entries from multiple log files). Returns an <a class="reference internal" href="../glossary.html#term-iterator"><span class="xref std std-term">iterator</span></a>
- over the sorted values.</p>
- <p>Similar to <code class="docutils literal notranslate"><span class="pre">sorted(itertools.chain(*iterables))</span></code> but returns an iterable, does
- not pull the data into memory all at once, and assumes that each of the input
- streams is already sorted (smallest to largest).</p>
- <p>Has two optional arguments which must be specified as keyword arguments.</p>
- <p><em>key</em> specifies a <a class="reference internal" href="../glossary.html#term-key-function"><span class="xref std std-term">key function</span></a> of one argument that is used to
- extract a comparison key from each input element. The default value is
- <code class="docutils literal notranslate"><span class="pre">None</span></code> (compare the elements directly).</p>
- <p><em>reverse</em> is a boolean value. If set to <code class="docutils literal notranslate"><span class="pre">True</span></code>, then the input elements
- are merged as if each comparison were reversed. To achieve behavior similar
- to <code class="docutils literal notranslate"><span class="pre">sorted(itertools.chain(*iterables),</span> <span class="pre">reverse=True)</span></code>, all iterables must
- be sorted from largest to smallest.</p>
- <div class="versionchanged">
- <p><span class="versionmodified changed">Changed in version 3.5: </span>Added the optional <em>key</em> and <em>reverse</em> parameters.</p>
- </div>
- </dd></dl>
-
- <dl class="py function">
- <dt class="sig sig-object py" id="heapq.nlargest">
- <span class="sig-prename descclassname"><span class="pre">heapq.</span></span><span class="sig-name descname"><span class="pre">nlargest</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">n</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">iterable</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">key</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">None</span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#heapq.nlargest" title="Link to this definition">¶</a></dt>
- <dd><p>Return a list with the <em>n</em> largest elements from the dataset defined by
- <em>iterable</em>. <em>key</em>, if provided, specifies a function of one argument that is
- used to extract a comparison key from each element in <em>iterable</em> (for example,
- <code class="docutils literal notranslate"><span class="pre">key=str.lower</span></code>). Equivalent to: <code class="docutils literal notranslate"><span class="pre">sorted(iterable,</span> <span class="pre">key=key,</span>
- <span class="pre">reverse=True)[:n]</span></code>.</p>
- </dd></dl>
-
- <dl class="py function">
- <dt class="sig sig-object py" id="heapq.nsmallest">
- <span class="sig-prename descclassname"><span class="pre">heapq.</span></span><span class="sig-name descname"><span class="pre">nsmallest</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">n</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">iterable</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">key</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">None</span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#heapq.nsmallest" title="Link to this definition">¶</a></dt>
- <dd><p>Return a list with the <em>n</em> smallest elements from the dataset defined by
- <em>iterable</em>. <em>key</em>, if provided, specifies a function of one argument that is
- used to extract a comparison key from each element in <em>iterable</em> (for example,
- <code class="docutils literal notranslate"><span class="pre">key=str.lower</span></code>). Equivalent to: <code class="docutils literal notranslate"><span class="pre">sorted(iterable,</span> <span class="pre">key=key)[:n]</span></code>.</p>
- </dd></dl>
-
- <p>The latter two functions perform best for smaller values of <em>n</em>. For larger
- values, it is more efficient to use the <a class="reference internal" href="functions.html#sorted" title="sorted"><code class="xref py py-func docutils literal notranslate"><span class="pre">sorted()</span></code></a> function. Also, when
- <code class="docutils literal notranslate"><span class="pre">n==1</span></code>, it is more efficient to use the built-in <a class="reference internal" href="functions.html#min" title="min"><code class="xref py py-func docutils literal notranslate"><span class="pre">min()</span></code></a> and <a class="reference internal" href="functions.html#max" title="max"><code class="xref py py-func docutils literal notranslate"><span class="pre">max()</span></code></a>
- functions. If repeated usage of these functions is required, consider turning
- the iterable into an actual heap.</p>
- <section id="basic-examples">
- <h2>Basic Examples<a class="headerlink" href="#basic-examples" title="Link to this heading">¶</a></h2>
- <p>A <a class="reference external" href="https://en.wikipedia.org/wiki/Heapsort">heapsort</a> can be implemented by
- pushing all values onto a heap and then popping off the smallest values one at a
- time:</p>
- <div class="highlight-python3 notranslate"><div class="highlight"><pre><span></span><span class="gp">>>> </span><span class="k">def</span> <span class="nf">heapsort</span><span class="p">(</span><span class="n">iterable</span><span class="p">):</span>
- <span class="gp">... </span> <span class="n">h</span> <span class="o">=</span> <span class="p">[]</span>
- <span class="gp">... </span> <span class="k">for</span> <span class="n">value</span> <span class="ow">in</span> <span class="n">iterable</span><span class="p">:</span>
- <span class="gp">... </span> <span class="n">heappush</span><span class="p">(</span><span class="n">h</span><span class="p">,</span> <span class="n">value</span><span class="p">)</span>
- <span class="gp">... </span> <span class="k">return</span> <span class="p">[</span><span class="n">heappop</span><span class="p">(</span><span class="n">h</span><span class="p">)</span> <span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="nb">len</span><span class="p">(</span><span class="n">h</span><span class="p">))]</span>
- <span class="gp">...</span>
- <span class="gp">>>> </span><span class="n">heapsort</span><span class="p">([</span><span class="mi">1</span><span class="p">,</span> <span class="mi">3</span><span class="p">,</span> <span class="mi">5</span><span class="p">,</span> <span class="mi">7</span><span class="p">,</span> <span class="mi">9</span><span class="p">,</span> <span class="mi">2</span><span class="p">,</span> <span class="mi">4</span><span class="p">,</span> <span class="mi">6</span><span class="p">,</span> <span class="mi">8</span><span class="p">,</span> <span class="mi">0</span><span class="p">])</span>
- <span class="go">[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]</span>
- </pre></div>
- </div>
- <p>This is similar to <code class="docutils literal notranslate"><span class="pre">sorted(iterable)</span></code>, but unlike <a class="reference internal" href="functions.html#sorted" title="sorted"><code class="xref py py-func docutils literal notranslate"><span class="pre">sorted()</span></code></a>, this
- implementation is not stable.</p>
- <p>Heap elements can be tuples. This is useful for assigning comparison values
- (such as task priorities) alongside the main record being tracked:</p>
- <div class="highlight-python3 notranslate"><div class="highlight"><pre><span></span><span class="gp">>>> </span><span class="n">h</span> <span class="o">=</span> <span class="p">[]</span>
- <span class="gp">>>> </span><span class="n">heappush</span><span class="p">(</span><span class="n">h</span><span class="p">,</span> <span class="p">(</span><span class="mi">5</span><span class="p">,</span> <span class="s1">'write code'</span><span class="p">))</span>
- <span class="gp">>>> </span><span class="n">heappush</span><span class="p">(</span><span class="n">h</span><span class="p">,</span> <span class="p">(</span><span class="mi">7</span><span class="p">,</span> <span class="s1">'release product'</span><span class="p">))</span>
- <span class="gp">>>> </span><span class="n">heappush</span><span class="p">(</span><span class="n">h</span><span class="p">,</span> <span class="p">(</span><span class="mi">1</span><span class="p">,</span> <span class="s1">'write spec'</span><span class="p">))</span>
- <span class="gp">>>> </span><span class="n">heappush</span><span class="p">(</span><span class="n">h</span><span class="p">,</span> <span class="p">(</span><span class="mi">3</span><span class="p">,</span> <span class="s1">'create tests'</span><span class="p">))</span>
- <span class="gp">>>> </span><span class="n">heappop</span><span class="p">(</span><span class="n">h</span><span class="p">)</span>
- <span class="go">(1, 'write spec')</span>
- </pre></div>
- </div>
- </section>
- <section id="priority-queue-implementation-notes">
- <h2>Priority Queue Implementation Notes<a class="headerlink" href="#priority-queue-implementation-notes" title="Link to this heading">¶</a></h2>
- <p>A <a class="reference external" href="https://en.wikipedia.org/wiki/Priority_queue">priority queue</a> is common use
- for a heap, and it presents several implementation challenges:</p>
- <ul class="simple">
- <li><p>Sort stability: how do you get two tasks with equal priorities to be returned
- in the order they were originally added?</p></li>
- <li><p>Tuple comparison breaks for (priority, task) pairs if the priorities are equal
- and the tasks do not have a default comparison order.</p></li>
- <li><p>If the priority of a task changes, how do you move it to a new position in
- the heap?</p></li>
- <li><p>Or if a pending task needs to be deleted, how do you find it and remove it
- from the queue?</p></li>
- </ul>
- <p>A solution to the first two challenges is to store entries as 3-element list
- including the priority, an entry count, and the task. The entry count serves as
- a tie-breaker so that two tasks with the same priority are returned in the order
- they were added. And since no two entry counts are the same, the tuple
- comparison will never attempt to directly compare two tasks.</p>
- <p>Another solution to the problem of non-comparable tasks is to create a wrapper
- class that ignores the task item and only compares the priority field:</p>
- <div class="highlight-python3 notranslate"><div class="highlight"><pre><span></span><span class="kn">from</span> <span class="nn">dataclasses</span> <span class="kn">import</span> <span class="n">dataclass</span><span class="p">,</span> <span class="n">field</span>
- <span class="kn">from</span> <span class="nn">typing</span> <span class="kn">import</span> <span class="n">Any</span>
-
- <span class="nd">@dataclass</span><span class="p">(</span><span class="n">order</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>
- <span class="k">class</span> <span class="nc">PrioritizedItem</span><span class="p">:</span>
- <span class="n">priority</span><span class="p">:</span> <span class="nb">int</span>
- <span class="n">item</span><span class="p">:</span> <span class="n">Any</span><span class="o">=</span><span class="n">field</span><span class="p">(</span><span class="n">compare</span><span class="o">=</span><span class="kc">False</span><span class="p">)</span>
- </pre></div>
- </div>
- <p>The remaining challenges revolve around finding a pending task and making
- changes to its priority or removing it entirely. Finding a task can be done
- with a dictionary pointing to an entry in the queue.</p>
- <p>Removing the entry or changing its priority is more difficult because it would
- break the heap structure invariants. So, a possible solution is to mark the
- entry as removed and add a new entry with the revised priority:</p>
- <div class="highlight-python3 notranslate"><div class="highlight"><pre><span></span><span class="n">pq</span> <span class="o">=</span> <span class="p">[]</span> <span class="c1"># list of entries arranged in a heap</span>
- <span class="n">entry_finder</span> <span class="o">=</span> <span class="p">{}</span> <span class="c1"># mapping of tasks to entries</span>
- <span class="n">REMOVED</span> <span class="o">=</span> <span class="s1">'<removed-task>'</span> <span class="c1"># placeholder for a removed task</span>
- <span class="n">counter</span> <span class="o">=</span> <span class="n">itertools</span><span class="o">.</span><span class="n">count</span><span class="p">()</span> <span class="c1"># unique sequence count</span>
-
- <span class="k">def</span> <span class="nf">add_task</span><span class="p">(</span><span class="n">task</span><span class="p">,</span> <span class="n">priority</span><span class="o">=</span><span class="mi">0</span><span class="p">):</span>
- <span class="s1">'Add a new task or update the priority of an existing task'</span>
- <span class="k">if</span> <span class="n">task</span> <span class="ow">in</span> <span class="n">entry_finder</span><span class="p">:</span>
- <span class="n">remove_task</span><span class="p">(</span><span class="n">task</span><span class="p">)</span>
- <span class="n">count</span> <span class="o">=</span> <span class="nb">next</span><span class="p">(</span><span class="n">counter</span><span class="p">)</span>
- <span class="n">entry</span> <span class="o">=</span> <span class="p">[</span><span class="n">priority</span><span class="p">,</span> <span class="n">count</span><span class="p">,</span> <span class="n">task</span><span class="p">]</span>
- <span class="n">entry_finder</span><span class="p">[</span><span class="n">task</span><span class="p">]</span> <span class="o">=</span> <span class="n">entry</span>
- <span class="n">heappush</span><span class="p">(</span><span class="n">pq</span><span class="p">,</span> <span class="n">entry</span><span class="p">)</span>
-
- <span class="k">def</span> <span class="nf">remove_task</span><span class="p">(</span><span class="n">task</span><span class="p">):</span>
- <span class="s1">'Mark an existing task as REMOVED. Raise KeyError if not found.'</span>
- <span class="n">entry</span> <span class="o">=</span> <span class="n">entry_finder</span><span class="o">.</span><span class="n">pop</span><span class="p">(</span><span class="n">task</span><span class="p">)</span>
- <span class="n">entry</span><span class="p">[</span><span class="o">-</span><span class="mi">1</span><span class="p">]</span> <span class="o">=</span> <span class="n">REMOVED</span>
-
- <span class="k">def</span> <span class="nf">pop_task</span><span class="p">():</span>
- <span class="s1">'Remove and return the lowest priority task. Raise KeyError if empty.'</span>
- <span class="k">while</span> <span class="n">pq</span><span class="p">:</span>
- <span class="n">priority</span><span class="p">,</span> <span class="n">count</span><span class="p">,</span> <span class="n">task</span> <span class="o">=</span> <span class="n">heappop</span><span class="p">(</span><span class="n">pq</span><span class="p">)</span>
- <span class="k">if</span> <span class="n">task</span> <span class="ow">is</span> <span class="ow">not</span> <span class="n">REMOVED</span><span class="p">:</span>
- <span class="k">del</span> <span class="n">entry_finder</span><span class="p">[</span><span class="n">task</span><span class="p">]</span>
- <span class="k">return</span> <span class="n">task</span>
- <span class="k">raise</span> <span class="ne">KeyError</span><span class="p">(</span><span class="s1">'pop from an empty priority queue'</span><span class="p">)</span>
- </pre></div>
- </div>
- </section>
- <section id="theory">
- <h2>Theory<a class="headerlink" href="#theory" title="Link to this heading">¶</a></h2>
- <p>Heaps are arrays for which <code class="docutils literal notranslate"><span class="pre">a[k]</span> <span class="pre"><=</span> <span class="pre">a[2*k+1]</span></code> and <code class="docutils literal notranslate"><span class="pre">a[k]</span> <span class="pre"><=</span> <span class="pre">a[2*k+2]</span></code> for all
- <em>k</em>, counting elements from 0. For the sake of comparison, non-existing
- elements are considered to be infinite. The interesting property of a heap is
- that <code class="docutils literal notranslate"><span class="pre">a[0]</span></code> is always its smallest element.</p>
- <p>The strange invariant above is meant to be an efficient memory representation
- for a tournament. The numbers below are <em>k</em>, not <code class="docutils literal notranslate"><span class="pre">a[k]</span></code>:</p>
- <div class="highlight-python3 notranslate"><div class="highlight"><pre><span></span> <span class="mi">0</span>
-
- <span class="mi">1</span> <span class="mi">2</span>
-
- <span class="mi">3</span> <span class="mi">4</span> <span class="mi">5</span> <span class="mi">6</span>
-
- <span class="mi">7</span> <span class="mi">8</span> <span class="mi">9</span> <span class="mi">10</span> <span class="mi">11</span> <span class="mi">12</span> <span class="mi">13</span> <span class="mi">14</span>
-
- <span class="mi">15</span> <span class="mi">16</span> <span class="mi">17</span> <span class="mi">18</span> <span class="mi">19</span> <span class="mi">20</span> <span class="mi">21</span> <span class="mi">22</span> <span class="mi">23</span> <span class="mi">24</span> <span class="mi">25</span> <span class="mi">26</span> <span class="mi">27</span> <span class="mi">28</span> <span class="mi">29</span> <span class="mi">30</span>
- </pre></div>
- </div>
- <p>In the tree above, each cell <em>k</em> is topping <code class="docutils literal notranslate"><span class="pre">2*k+1</span></code> and <code class="docutils literal notranslate"><span class="pre">2*k+2</span></code>. In a usual
- binary tournament we see in sports, each cell is the winner over the two cells
- it tops, and we can trace the winner down the tree to see all opponents s/he
- had. However, in many computer applications of such tournaments, we do not need
- to trace the history of a winner. To be more memory efficient, when a winner is
- promoted, we try to replace it by something else at a lower level, and the rule
- becomes that a cell and the two cells it tops contain three different items, but
- the top cell “wins” over the two topped cells.</p>
- <p>If this heap invariant is protected at all time, index 0 is clearly the overall
- winner. The simplest algorithmic way to remove it and find the “next” winner is
- to move some loser (let’s say cell 30 in the diagram above) into the 0 position,
- and then percolate this new 0 down the tree, exchanging values, until the
- invariant is re-established. This is clearly logarithmic on the total number of
- items in the tree. By iterating over all items, you get an <em>O</em>(<em>n</em> log <em>n</em>) sort.</p>
- <p>A nice feature of this sort is that you can efficiently insert new items while
- the sort is going on, provided that the inserted items are not “better” than the
- last 0’th element you extracted. This is especially useful in simulation
- contexts, where the tree holds all incoming events, and the “win” condition
- means the smallest scheduled time. When an event schedules other events for
- execution, they are scheduled into the future, so they can easily go into the
- heap. So, a heap is a good structure for implementing schedulers (this is what
- I used for my MIDI sequencer :-).</p>
- <p>Various structures for implementing schedulers have been extensively studied,
- and heaps are good for this, as they are reasonably speedy, the speed is almost
- constant, and the worst case is not much different than the average case.
- However, there are other representations which are more efficient overall, yet
- the worst cases might be terrible.</p>
- <p>Heaps are also very useful in big disk sorts. You most probably all know that a
- big sort implies producing “runs” (which are pre-sorted sequences, whose size is
- usually related to the amount of CPU memory), followed by a merging passes for
- these runs, which merging is often very cleverly organised <a class="footnote-reference brackets" href="#id2" id="id1" role="doc-noteref"><span class="fn-bracket">[</span>1<span class="fn-bracket">]</span></a>. It is very
- important that the initial sort produces the longest runs possible. Tournaments
- are a good way to achieve that. If, using all the memory available to hold a
- tournament, you replace and percolate items that happen to fit the current run,
- you’ll produce runs which are twice the size of the memory for random input, and
- much better for input fuzzily ordered.</p>
- <p>Moreover, if you output the 0’th item on disk and get an input which may not fit
- in the current tournament (because the value “wins” over the last output value),
- it cannot fit in the heap, so the size of the heap decreases. The freed memory
- could be cleverly reused immediately for progressively building a second heap,
- which grows at exactly the same rate the first heap is melting. When the first
- heap completely vanishes, you switch heaps and start a new run. Clever and
- quite effective!</p>
- <p>In a word, heaps are useful memory structures to know. I use them in a few
- applications, and I think it is good to keep a ‘heap’ module around. :-)</p>
- <p class="rubric">Footnotes</p>
- <aside class="footnote-list brackets">
- <aside class="footnote brackets" id="id2" role="doc-footnote">
- <span class="label"><span class="fn-bracket">[</span><a role="doc-backlink" href="#id1">1</a><span class="fn-bracket">]</span></span>
- <p>The disk balancing algorithms which are current, nowadays, are more annoying
- than clever, and this is a consequence of the seeking capabilities of the disks.
- On devices which cannot seek, like big tape drives, the story was quite
- different, and one had to be very clever to ensure (far in advance) that each
- tape movement will be the most effective possible (that is, will best
- participate at “progressing” the merge). Some tapes were even able to read
- backwards, and this was also used to avoid the rewinding time. Believe me, real
- good tape sorts were quite spectacular to watch! From all times, sorting has
- always been a Great Art! :-)</p>
- </aside>
- </aside>
- </section>
- </section>
-
-
- <div class="clearer"></div>
- </div>
- </div>
- </div>
- <div class="sphinxsidebar" role="navigation" aria-label="main navigation">
- <div class="sphinxsidebarwrapper">
- <div>
- <h3><a href="../contents.html">Table of Contents</a></h3>
- <ul>
- <li><a class="reference internal" href="#"><code class="xref py py-mod docutils literal notranslate"><span class="pre">heapq</span></code> — Heap queue algorithm</a><ul>
- <li><a class="reference internal" href="#basic-examples">Basic Examples</a></li>
- <li><a class="reference internal" href="#priority-queue-implementation-notes">Priority Queue Implementation Notes</a></li>
- <li><a class="reference internal" href="#theory">Theory</a></li>
- </ul>
- </li>
- </ul>
-
- </div>
- <div>
- <h4>Previous topic</h4>
- <p class="topless"><a href="collections.abc.html"
- title="previous chapter"><code class="xref py py-mod docutils literal notranslate"><span class="pre">collections.abc</span></code> — Abstract Base Classes for Containers</a></p>
- </div>
- <div>
- <h4>Next topic</h4>
- <p class="topless"><a href="bisect.html"
- title="next chapter"><code class="xref py py-mod docutils literal notranslate"><span class="pre">bisect</span></code> — Array bisection algorithm</a></p>
- </div>
- <div role="note" aria-label="source link">
- <h3>This Page</h3>
- <ul class="this-page-menu">
- <li><a href="../bugs.html">Report a Bug</a></li>
- <li>
- <a href="https://github.com/python/cpython/blob/main/Doc/library/heapq.rst"
- rel="nofollow">Show Source
- </a>
- </li>
- </ul>
- </div>
- </div>
- <div id="sidebarbutton" title="Collapse sidebar">
- <span>«</span>
- </div>
-
- </div>
- <div class="clearer"></div>
- </div>
- <div class="related" role="navigation" aria-label="related navigation">
- <h3>Navigation</h3>
- <ul>
- <li class="right" style="margin-right: 10px">
- <a href="../genindex.html" title="General Index"
- >index</a></li>
- <li class="right" >
- <a href="../py-modindex.html" title="Python Module Index"
- >modules</a> |</li>
- <li class="right" >
- <a href="bisect.html" title="bisect — Array bisection algorithm"
- >next</a> |</li>
- <li class="right" >
- <a href="collections.abc.html" title="collections.abc — Abstract Base Classes for Containers"
- >previous</a> |</li>
-
- <li><img src="../_static/py.svg" alt="Python logo" style="vertical-align: middle; margin-top: -1px"/></li>
- <li><a href="https://www.python.org/">Python</a> »</li>
- <li class="switchers">
- <div class="language_switcher_placeholder"></div>
- <div class="version_switcher_placeholder"></div>
- </li>
- <li>
-
- </li>
- <li id="cpython-language-and-version">
- <a href="../index.html">3.12.3 Documentation</a> »
- </li>
-
- <li class="nav-item nav-item-1"><a href="index.html" >The Python Standard Library</a> »</li>
- <li class="nav-item nav-item-2"><a href="datatypes.html" >Data Types</a> »</li>
- <li class="nav-item nav-item-this"><a href=""><code class="xref py py-mod docutils literal notranslate"><span class="pre">heapq</span></code> — Heap queue algorithm</a></li>
- <li class="right">
-
-
- <div class="inline-search" role="search">
- <form class="inline-search" action="../search.html" method="get">
- <input placeholder="Quick search" aria-label="Quick search" type="search" name="q" id="search-box" />
- <input type="submit" value="Go" />
- </form>
- </div>
- |
- </li>
- <li class="right">
- <label class="theme-selector-label">
- Theme
- <select class="theme-selector" oninput="activateTheme(this.value)">
- <option value="auto" selected>Auto</option>
- <option value="light">Light</option>
- <option value="dark">Dark</option>
- </select>
- </label> |</li>
-
- </ul>
- </div>
- <div class="footer">
- ©
- <a href="../copyright.html">
-
- Copyright
-
- </a>
- 2001-2024, Python Software Foundation.
- <br />
- This page is licensed under the Python Software Foundation License Version 2.
- <br />
- Examples, recipes, and other code in the documentation are additionally licensed under the Zero Clause BSD License.
- <br />
-
- See <a href="/license.html">History and License</a> for more information.<br />
-
-
- <br />
-
- The Python Software Foundation is a non-profit corporation.
- <a href="https://www.python.org/psf/donations/">Please donate.</a>
- <br />
- <br />
- Last updated on Apr 09, 2024 (13:47 UTC).
-
- <a href="/bugs.html">Found a bug</a>?
-
- <br />
-
- Created using <a href="https://www.sphinx-doc.org/">Sphinx</a> 7.2.6.
- </div>
-
- </body>
- </html>
|