Baoding-FenShuaJiang
/
Green_GoodERP18_FSJ_Standard
Seguir 1
Destacar 0
Wiki 捐赠
gooderp18绿色标准版
No puede seleccionar más de 25 temas Los temas deben comenzar con una letra o número, pueden incluir guiones ('-') y pueden tener hasta 35 caracteres de largo.
Rama: master
Ramas Etiquetas
${ item.name }
Crear rama ${ searchTerm }
desde 'master'
${ noResults }
Green_GoodERP18_FSJ_Standard/python/Doc/html/library/pickle.html

1562 líneas
162KB
Original Enlace permanente Blame Histórico 

  1. <!DOCTYPE html>

  2. 

  3. <html lang="en" data-content_root="../">

  4.   <head>

  5.     <meta charset="utf-8" />

  6.     <meta name="viewport" content="width=device-width, initial-scale=1.0" /><meta name="viewport" content="width=device-width, initial-scale=1" />

  7. <meta property="og:title" content="pickle — Python object serialization" />

  8. <meta property="og:type" content="website" />

  9. <meta property="og:url" content="https://docs.python.org/3/library/pickle.html" />

  10. <meta property="og:site_name" content="Python documentation" />

  11. <meta property="og:description" content="Source code: Lib/pickle.py The pickle module implements binary protocols for serializing and de-serializing a Python object structure. “Pickling” is the process whereby a Python object hierarchy is..." />

  12. <meta property="og:image" content="https://docs.python.org/3/_static/og-image.png" />

  13. <meta property="og:image:alt" content="Python documentation" />

  14. <meta name="description" content="Source code: Lib/pickle.py The pickle module implements binary protocols for serializing and de-serializing a Python object structure. “Pickling” is the process whereby a Python object hierarchy is..." />

  15. <meta property="og:image:width" content="200" />

  16. <meta property="og:image:height" content="200" />

  17. <meta name="theme-color" content="#3776ab" />

  18. 

  19.     <title>pickle — Python object serialization &#8212; Python 3.12.3 documentation</title><meta name="viewport" content="width=device-width, initial-scale=1.0">

  20.     

  21.     <link rel="stylesheet" type="text/css" href="../_static/pygments.css?v=80d5e7a1" />

  22.     <link rel="stylesheet" type="text/css" href="../_static/pydoctheme.css?v=bb723527" />

  23.     <link id="pygments_dark_css" media="(prefers-color-scheme: dark)" rel="stylesheet" type="text/css" href="../_static/pygments_dark.css?v=b20cc3f5" />

  24.     

  25.     <script src="../_static/documentation_options.js?v=2c828074"></script>

  26.     <script src="../_static/doctools.js?v=888ff710"></script>

  27.     <script src="../_static/sphinx_highlight.js?v=dc90522c"></script>

  28.     

  29.     <script src="../_static/sidebar.js"></script>

  30.     

  31.     <link rel="search" type="application/opensearchdescription+xml"

  32.           title="Search within Python 3.12.3 documentation"

  33.           href="../_static/opensearch.xml"/>

  34.     <link rel="author" title="About these documents" href="../about.html" />

  35.     <link rel="index" title="Index" href="../genindex.html" />

  36.     <link rel="search" title="Search" href="../search.html" />

  37.     <link rel="copyright" title="Copyright" href="../copyright.html" />

  38.     <link rel="next" title="copyreg — Register pickle support functions" href="copyreg.html" />

  39.     <link rel="prev" title="Data Persistence" href="persistence.html" />

  40.     <link rel="canonical" href="https://docs.python.org/3/library/pickle.html" />

  41.     

  42.       

  43.     

  44. 

  45.     

  46.     <style>

  47.       @media only screen {

  48.         table.full-width-table {

  49.             width: 100%;

  50.         }

  51.       }

  52.     </style>

  53. <link rel="stylesheet" href="../_static/pydoctheme_dark.css" media="(prefers-color-scheme: dark)" id="pydoctheme_dark_css">

  54.     <link rel="shortcut icon" type="image/png" href="../_static/py.svg" />

  55.             <script type="text/javascript" src="../_static/copybutton.js"></script>

  56.             <script type="text/javascript" src="../_static/menu.js"></script>

  57.             <script type="text/javascript" src="../_static/search-focus.js"></script>

  58.             <script type="text/javascript" src="../_static/themetoggle.js"></script> 

  59. 

  60.   </head>

  61. <body>

  62. <div class="mobile-nav">

  63.     <input type="checkbox" id="menuToggler" class="toggler__input" aria-controls="navigation"

  64.            aria-pressed="false" aria-expanded="false" role="button" aria-label="Menu" />

  65.     <nav class="nav-content" role="navigation">

  66.         <label for="menuToggler" class="toggler__label">

  67.             <span></span>

  68.         </label>

  69.         <span class="nav-items-wrapper">

  70.             <a href="https://www.python.org/" class="nav-logo">

  71.                 <img src="../_static/py.svg" alt="Python logo"/>

  72.             </a>

  73.             <span class="version_switcher_placeholder"></span>

  74.             <form role="search" class="search" action="../search.html" method="get">

  75.                 <svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" viewBox="0 0 24 24" class="search-icon">

  76.                     <path fill-rule="nonzero" fill="currentColor" d="M15.5 14h-.79l-.28-.27a6.5 6.5 0 001.48-5.34c-.47-2.78-2.79-5-5.59-5.34a6.505 6.505 0 00-7.27 7.27c.34 2.8 2.56 5.12 5.34 5.59a6.5 6.5 0 005.34-1.48l.27.28v.79l4.25 4.25c.41.41 1.08.41 1.49 0 .41-.41.41-1.08 0-1.49L15.5 14zm-6 0C7.01 14 5 11.99 5 9.5S7.01 5 9.5 5 14 7.01 14 9.5 11.99 14 9.5 14z"></path>

  77.                 </svg>

  78.                 <input placeholder="Quick search" aria-label="Quick search" type="search" name="q" />

  79.                 <input type="submit" value="Go"/>

  80.             </form>

  81.         </span>

  82.     </nav>

  83.     <div class="menu-wrapper">

  84.         <nav class="menu" role="navigation" aria-label="main navigation">

  85.             <div class="language_switcher_placeholder"></div>

  86.             

  87. <label class="theme-selector-label">

  88.     Theme

  89.     <select class="theme-selector" oninput="activateTheme(this.value)">

  90.         <option value="auto" selected>Auto</option>

  91.         <option value="light">Light</option>

  92.         <option value="dark">Dark</option>

  93.     </select>

  94. </label>

  95.   <div>

  96.     <h3><a href="../contents.html">Table of Contents</a></h3>

  97.     <ul>

  98. <li><a class="reference internal" href="#"><code class="xref py py-mod docutils literal notranslate"><span class="pre">pickle</span></code> — Python object serialization</a><ul>

  99. <li><a class="reference internal" href="#relationship-to-other-python-modules">Relationship to other Python modules</a><ul>

  100. <li><a class="reference internal" href="#comparison-with-marshal">Comparison with <code class="docutils literal notranslate"><span class="pre">marshal</span></code></a></li>

  101. <li><a class="reference internal" href="#comparison-with-json">Comparison with <code class="docutils literal notranslate"><span class="pre">json</span></code></a></li>

  102. </ul>

  103. </li>

  104. <li><a class="reference internal" href="#data-stream-format">Data stream format</a></li>

  105. <li><a class="reference internal" href="#module-interface">Module Interface</a></li>

  106. <li><a class="reference internal" href="#what-can-be-pickled-and-unpickled">What can be pickled and unpickled?</a></li>

  107. <li><a class="reference internal" href="#pickling-class-instances">Pickling Class Instances</a><ul>

  108. <li><a class="reference internal" href="#persistence-of-external-objects">Persistence of External Objects</a></li>

  109. <li><a class="reference internal" href="#dispatch-tables">Dispatch Tables</a></li>

  110. <li><a class="reference internal" href="#handling-stateful-objects">Handling Stateful Objects</a></li>

  111. </ul>

  112. </li>

  113. <li><a class="reference internal" href="#custom-reduction-for-types-functions-and-other-objects">Custom Reduction for Types, Functions, and Other Objects</a></li>

  114. <li><a class="reference internal" href="#out-of-band-buffers">Out-of-band Buffers</a><ul>

  115. <li><a class="reference internal" href="#provider-api">Provider API</a></li>

  116. <li><a class="reference internal" href="#consumer-api">Consumer API</a></li>

  117. <li><a class="reference internal" href="#example">Example</a></li>

  118. </ul>

  119. </li>

  120. <li><a class="reference internal" href="#restricting-globals">Restricting Globals</a></li>

  121. <li><a class="reference internal" href="#performance">Performance</a></li>

  122. <li><a class="reference internal" href="#examples">Examples</a></li>

  123. </ul>

  124. </li>

  125. </ul>

  126. 

  127.   </div>

  128.   <div>

  129.     <h4>Previous topic</h4>

  130.     <p class="topless"><a href="persistence.html"

  131.                           title="previous chapter">Data Persistence</a></p>

  132.   </div>

  133.   <div>

  134.     <h4>Next topic</h4>

  135.     <p class="topless"><a href="copyreg.html"

  136.                           title="next chapter"><code class="xref py py-mod docutils literal notranslate"><span class="pre">copyreg</span></code> — Register <code class="xref py py-mod docutils literal notranslate"><span class="pre">pickle</span></code> support functions</a></p>

  137.   </div>

  138.   <div role="note" aria-label="source link">

  139.     <h3>This Page</h3>

  140.     <ul class="this-page-menu">

  141.       <li><a href="../bugs.html">Report a Bug</a></li>

  142.       <li>

  143.         <a href="https://github.com/python/cpython/blob/main/Doc/library/pickle.rst"

  144.             rel="nofollow">Show Source

  145.         </a>

  146.       </li>

  147.     </ul>

  148.   </div>

  149.         </nav>

  150.     </div>

  151. </div>

  152. 

  153.   

  154.     <div class="related" role="navigation" aria-label="related navigation">

  155.       <h3>Navigation</h3>

  156.       <ul>

  157.         <li class="right" style="margin-right: 10px">

  158.           <a href="../genindex.html" title="General Index"

  159.              accesskey="I">index</a></li>

  160.         <li class="right" >

  161.           <a href="../py-modindex.html" title="Python Module Index"

  162.              >modules</a> |</li>

  163.         <li class="right" >

  164.           <a href="copyreg.html" title="copyreg — Register pickle support functions"

  165.              accesskey="N">next</a> |</li>

  166.         <li class="right" >

  167.           <a href="persistence.html" title="Data Persistence"

  168.              accesskey="P">previous</a> |</li>

  169. 

  170.           <li><img src="../_static/py.svg" alt="Python logo" style="vertical-align: middle; margin-top: -1px"/></li>

  171.           <li><a href="https://www.python.org/">Python</a> &#187;</li>

  172.           <li class="switchers">

  173.             <div class="language_switcher_placeholder"></div>

  174.             <div class="version_switcher_placeholder"></div>

  175.           </li>

  176.           <li>

  177.               

  178.           </li>

  179.     <li id="cpython-language-and-version">

  180.       <a href="../index.html">3.12.3 Documentation</a> &#187;

  181.     </li>

  182. 

  183.           <li class="nav-item nav-item-1"><a href="index.html" >The Python Standard Library</a> &#187;</li>

  184.           <li class="nav-item nav-item-2"><a href="persistence.html" accesskey="U">Data Persistence</a> &#187;</li>

  185.         <li class="nav-item nav-item-this"><a href=""><code class="xref py py-mod docutils literal notranslate"><span class="pre">pickle</span></code> — Python object serialization</a></li>

  186.                 <li class="right">

  187.                     

  188. 

  189.     <div class="inline-search" role="search">

  190.         <form class="inline-search" action="../search.html" method="get">

  191.           <input placeholder="Quick search" aria-label="Quick search" type="search" name="q" id="search-box" />

  192.           <input type="submit" value="Go" />

  193.         </form>

  194.     </div>

  195.                      |

  196.                 </li>

  197.             <li class="right">

  198. <label class="theme-selector-label">

  199.     Theme

  200.     <select class="theme-selector" oninput="activateTheme(this.value)">

  201.         <option value="auto" selected>Auto</option>

  202.         <option value="light">Light</option>

  203.         <option value="dark">Dark</option>

  204.     </select>

  205. </label> |</li>

  206.             

  207.       </ul>

  208.     </div>    

  209. 

  210.     <div class="document">

  211.       <div class="documentwrapper">

  212.         <div class="bodywrapper">

  213.           <div class="body" role="main">

  214.             

  215.   <section id="module-pickle">

  216. <span id="pickle-python-object-serialization"></span><h1><a class="reference internal" href="#module-pickle" title="pickle: Convert Python objects to streams of bytes and back."><code class="xref py py-mod docutils literal notranslate"><span class="pre">pickle</span></code></a> — Python object serialization<a class="headerlink" href="#module-pickle" title="Link to this heading">¶</a></h1>

  217. <p><strong>Source code:</strong> <a class="reference external" href="https://github.com/python/cpython/tree/3.12/Lib/pickle.py">Lib/pickle.py</a></p>

  218. <hr class="docutils" id="index-0" />

  219. <p>The <a class="reference internal" href="#module-pickle" title="pickle: Convert Python objects to streams of bytes and back."><code class="xref py py-mod docutils literal notranslate"><span class="pre">pickle</span></code></a> module implements binary protocols for serializing and

  220. de-serializing a Python object structure.  <em>“Pickling”</em> is the process

  221. whereby a Python object hierarchy is converted into a byte stream, and

  222. <em>“unpickling”</em> is the inverse operation, whereby a byte stream

  223. (from a <a class="reference internal" href="../glossary.html#term-binary-file"><span class="xref std std-term">binary file</span></a> or <a class="reference internal" href="../glossary.html#term-bytes-like-object"><span class="xref std std-term">bytes-like object</span></a>) is converted

  224. back into an object hierarchy.  Pickling (and unpickling) is alternatively

  225. known as “serialization”, “marshalling,” <a class="footnote-reference brackets" href="#id7" id="id1" role="doc-noteref"><span class="fn-bracket">[</span>1<span class="fn-bracket">]</span></a> or “flattening”; however, to

  226. avoid confusion, the terms used here are “pickling” and “unpickling”.</p>

  227. <div class="admonition warning">

  228. <p class="admonition-title">Warning</p>

  229. <p>The <code class="docutils literal notranslate"><span class="pre">pickle</span></code> module <strong>is not secure</strong>. Only unpickle data you trust.</p>

  230. <p>It is possible to construct malicious pickle data which will <strong>execute

  231. arbitrary code during unpickling</strong>. Never unpickle data that could have come

  232. from an untrusted source, or that could have been tampered with.</p>

  233. <p>Consider signing data with <a class="reference internal" href="hmac.html#module-hmac" title="hmac: Keyed-Hashing for Message Authentication (HMAC) implementation"><code class="xref py py-mod docutils literal notranslate"><span class="pre">hmac</span></code></a> if you need to ensure that it has not

  234. been tampered with.</p>

  235. <p>Safer serialization formats such as <a class="reference internal" href="json.html#module-json" title="json: Encode and decode the JSON format."><code class="xref py py-mod docutils literal notranslate"><span class="pre">json</span></code></a> may be more appropriate if

  236. you are processing untrusted data. See <a class="reference internal" href="#comparison-with-json"><span class="std std-ref">Comparison with json</span></a>.</p>

  237. </div>

  238. <section id="relationship-to-other-python-modules">

  239. <h2>Relationship to other Python modules<a class="headerlink" href="#relationship-to-other-python-modules" title="Link to this heading">¶</a></h2>

  240. <section id="comparison-with-marshal">

  241. <h3>Comparison with <code class="docutils literal notranslate"><span class="pre">marshal</span></code><a class="headerlink" href="#comparison-with-marshal" title="Link to this heading">¶</a></h3>

  242. <p>Python has a more primitive serialization module called <a class="reference internal" href="marshal.html#module-marshal" title="marshal: Convert Python objects to streams of bytes and back (with different constraints)."><code class="xref py py-mod docutils literal notranslate"><span class="pre">marshal</span></code></a>, but in

  243. general <a class="reference internal" href="#module-pickle" title="pickle: Convert Python objects to streams of bytes and back."><code class="xref py py-mod docutils literal notranslate"><span class="pre">pickle</span></code></a> should always be the preferred way to serialize Python

  244. objects.  <a class="reference internal" href="marshal.html#module-marshal" title="marshal: Convert Python objects to streams of bytes and back (with different constraints)."><code class="xref py py-mod docutils literal notranslate"><span class="pre">marshal</span></code></a> exists primarily to support Python’s <code class="file docutils literal notranslate"><span class="pre">.pyc</span></code>

  245. files.</p>

  246. <p>The <a class="reference internal" href="#module-pickle" title="pickle: Convert Python objects to streams of bytes and back."><code class="xref py py-mod docutils literal notranslate"><span class="pre">pickle</span></code></a> module differs from <a class="reference internal" href="marshal.html#module-marshal" title="marshal: Convert Python objects to streams of bytes and back (with different constraints)."><code class="xref py py-mod docutils literal notranslate"><span class="pre">marshal</span></code></a> in several significant ways:</p>

  247. <ul>

  248. <li><p>The <a class="reference internal" href="#module-pickle" title="pickle: Convert Python objects to streams of bytes and back."><code class="xref py py-mod docutils literal notranslate"><span class="pre">pickle</span></code></a> module keeps track of the objects it has already serialized,

  249. so that later references to the same object won’t be serialized again.

  250. <a class="reference internal" href="marshal.html#module-marshal" title="marshal: Convert Python objects to streams of bytes and back (with different constraints)."><code class="xref py py-mod docutils literal notranslate"><span class="pre">marshal</span></code></a> doesn’t do this.</p>

  251. <p>This has implications both for recursive objects and object sharing.  Recursive

  252. objects are objects that contain references to themselves.  These are not

  253. handled by marshal, and in fact, attempting to marshal recursive objects will

  254. crash your Python interpreter.  Object sharing happens when there are multiple

  255. references to the same object in different places in the object hierarchy being

  256. serialized.  <a class="reference internal" href="#module-pickle" title="pickle: Convert Python objects to streams of bytes and back."><code class="xref py py-mod docutils literal notranslate"><span class="pre">pickle</span></code></a> stores such objects only once, and ensures that all

  257. other references point to the master copy.  Shared objects remain shared, which

  258. can be very important for mutable objects.</p>

  259. </li>

  260. <li><p><a class="reference internal" href="marshal.html#module-marshal" title="marshal: Convert Python objects to streams of bytes and back (with different constraints)."><code class="xref py py-mod docutils literal notranslate"><span class="pre">marshal</span></code></a> cannot be used to serialize user-defined classes and their

  261. instances.  <a class="reference internal" href="#module-pickle" title="pickle: Convert Python objects to streams of bytes and back."><code class="xref py py-mod docutils literal notranslate"><span class="pre">pickle</span></code></a> can save and restore class instances transparently,

  262. however the class definition must be importable and live in the same module as

  263. when the object was stored.</p></li>

  264. <li><p>The <a class="reference internal" href="marshal.html#module-marshal" title="marshal: Convert Python objects to streams of bytes and back (with different constraints)."><code class="xref py py-mod docutils literal notranslate"><span class="pre">marshal</span></code></a> serialization format is not guaranteed to be portable

  265. across Python versions.  Because its primary job in life is to support

  266. <code class="file docutils literal notranslate"><span class="pre">.pyc</span></code> files, the Python implementers reserve the right to change the

  267. serialization format in non-backwards compatible ways should the need arise.

  268. The <a class="reference internal" href="#module-pickle" title="pickle: Convert Python objects to streams of bytes and back."><code class="xref py py-mod docutils literal notranslate"><span class="pre">pickle</span></code></a> serialization format is guaranteed to be backwards compatible

  269. across Python releases provided a compatible pickle protocol is chosen and

  270. pickling and unpickling code deals with Python 2 to Python 3 type differences

  271. if your data is crossing that unique breaking change language boundary.</p></li>

  272. </ul>

  273. </section>

  274. <section id="comparison-with-json">

  275. <span id="id2"></span><h3>Comparison with <code class="docutils literal notranslate"><span class="pre">json</span></code><a class="headerlink" href="#comparison-with-json" title="Link to this heading">¶</a></h3>

  276. <p>There are fundamental differences between the pickle protocols and

  277. <a class="reference external" href="https://json.org">JSON (JavaScript Object Notation)</a>:</p>

  278. <ul class="simple">

  279. <li><p>JSON is a text serialization format (it outputs unicode text, although

  280. most of the time it is then encoded to <code class="docutils literal notranslate"><span class="pre">utf-8</span></code>), while pickle is

  281. a binary serialization format;</p></li>

  282. <li><p>JSON is human-readable, while pickle is not;</p></li>

  283. <li><p>JSON is interoperable and widely used outside of the Python ecosystem,

  284. while pickle is Python-specific;</p></li>

  285. <li><p>JSON, by default, can only represent a subset of the Python built-in

  286. types, and no custom classes; pickle can represent an extremely large

  287. number of Python types (many of them automatically, by clever usage

  288. of Python’s introspection facilities; complex cases can be tackled by

  289. implementing <a class="reference internal" href="#pickle-inst"><span class="std std-ref">specific object APIs</span></a>);</p></li>

  290. <li><p>Unlike pickle, deserializing untrusted JSON does not in itself create an

  291. arbitrary code execution vulnerability.</p></li>

  292. </ul>

  293. <div class="admonition seealso">

  294. <p class="admonition-title">See also</p>

  295. <p>The <a class="reference internal" href="json.html#module-json" title="json: Encode and decode the JSON format."><code class="xref py py-mod docutils literal notranslate"><span class="pre">json</span></code></a> module: a standard library module allowing JSON

  296. serialization and deserialization.</p>

  297. </div>

  298. </section>

  299. </section>

  300. <section id="data-stream-format">

  301. <span id="pickle-protocols"></span><h2>Data stream format<a class="headerlink" href="#data-stream-format" title="Link to this heading">¶</a></h2>

  302. <p id="index-1">The data format used by <a class="reference internal" href="#module-pickle" title="pickle: Convert Python objects to streams of bytes and back."><code class="xref py py-mod docutils literal notranslate"><span class="pre">pickle</span></code></a> is Python-specific.  This has the

  303. advantage that there are no restrictions imposed by external standards such as

  304. JSON or XDR (which can’t represent pointer sharing); however it means that

  305. non-Python programs may not be able to reconstruct pickled Python objects.</p>

  306. <p>By default, the <a class="reference internal" href="#module-pickle" title="pickle: Convert Python objects to streams of bytes and back."><code class="xref py py-mod docutils literal notranslate"><span class="pre">pickle</span></code></a> data format uses a relatively compact binary

  307. representation.  If you need optimal size characteristics, you can efficiently

  308. <a class="reference internal" href="archiving.html"><span class="doc">compress</span></a> pickled data.</p>

  309. <p>The module <a class="reference internal" href="pickletools.html#module-pickletools" title="pickletools: Contains extensive comments about the pickle protocols and pickle-machine opcodes, as well as some useful functions."><code class="xref py py-mod docutils literal notranslate"><span class="pre">pickletools</span></code></a> contains tools for analyzing data streams

  310. generated by <a class="reference internal" href="#module-pickle" title="pickle: Convert Python objects to streams of bytes and back."><code class="xref py py-mod docutils literal notranslate"><span class="pre">pickle</span></code></a>.  <a class="reference internal" href="pickletools.html#module-pickletools" title="pickletools: Contains extensive comments about the pickle protocols and pickle-machine opcodes, as well as some useful functions."><code class="xref py py-mod docutils literal notranslate"><span class="pre">pickletools</span></code></a> source code has extensive

  311. comments about opcodes used by pickle protocols.</p>

  312. <p>There are currently 6 different protocols which can be used for pickling.

  313. The higher the protocol used, the more recent the version of Python needed

  314. to read the pickle produced.</p>

  315. <ul class="simple">

  316. <li><p>Protocol version 0 is the original “human-readable” protocol and is

  317. backwards compatible with earlier versions of Python.</p></li>

  318. <li><p>Protocol version 1 is an old binary format which is also compatible with

  319. earlier versions of Python.</p></li>

  320. <li><p>Protocol version 2 was introduced in Python 2.3.  It provides much more

  321. efficient pickling of <a class="reference internal" href="../glossary.html#term-new-style-class"><span class="xref std std-term">new-style classes</span></a>.  Refer to <span class="target" id="index-2"></span><a class="pep reference external" href="https://peps.python.org/pep-0307/"><strong>PEP 307</strong></a> for

  322. information about improvements brought by protocol 2.</p></li>

  323. <li><p>Protocol version 3 was added in Python 3.0.  It has explicit support for

  324. <a class="reference internal" href="stdtypes.html#bytes" title="bytes"><code class="xref py py-class docutils literal notranslate"><span class="pre">bytes</span></code></a> objects and cannot be unpickled by Python 2.x.  This was

  325. the default protocol in Python 3.0–3.7.</p></li>

  326. <li><p>Protocol version 4 was added in Python 3.4.  It adds support for very large

  327. objects, pickling more kinds of objects, and some data format

  328. optimizations.  It is the default protocol starting with Python 3.8.

  329. Refer to <span class="target" id="index-3"></span><a class="pep reference external" href="https://peps.python.org/pep-3154/"><strong>PEP 3154</strong></a> for information about improvements brought by

  330. protocol 4.</p></li>

  331. <li><p>Protocol version 5 was added in Python 3.8.  It adds support for out-of-band

  332. data and speedup for in-band data.  Refer to <span class="target" id="index-4"></span><a class="pep reference external" href="https://peps.python.org/pep-0574/"><strong>PEP 574</strong></a> for information about

  333. improvements brought by protocol 5.</p></li>

  334. </ul>

  335. <div class="admonition note">

  336. <p class="admonition-title">Note</p>

  337. <p>Serialization is a more primitive notion than persistence; although

  338. <a class="reference internal" href="#module-pickle" title="pickle: Convert Python objects to streams of bytes and back."><code class="xref py py-mod docutils literal notranslate"><span class="pre">pickle</span></code></a> reads and writes file objects, it does not handle the issue of

  339. naming persistent objects, nor the (even more complicated) issue of concurrent

  340. access to persistent objects.  The <a class="reference internal" href="#module-pickle" title="pickle: Convert Python objects to streams of bytes and back."><code class="xref py py-mod docutils literal notranslate"><span class="pre">pickle</span></code></a> module can transform a complex

  341. object into a byte stream and it can transform the byte stream into an object

  342. with the same internal structure.  Perhaps the most obvious thing to do with

  343. these byte streams is to write them onto a file, but it is also conceivable to

  344. send them across a network or store them in a database.  The <a class="reference internal" href="shelve.html#module-shelve" title="shelve: Python object persistence."><code class="xref py py-mod docutils literal notranslate"><span class="pre">shelve</span></code></a>

  345. module provides a simple interface to pickle and unpickle objects on

  346. DBM-style database files.</p>

  347. </div>

  348. </section>

  349. <section id="module-interface">

  350. <h2>Module Interface<a class="headerlink" href="#module-interface" title="Link to this heading">¶</a></h2>

  351. <p>To serialize an object hierarchy, you simply call the <a class="reference internal" href="#pickle.dumps" title="pickle.dumps"><code class="xref py py-func docutils literal notranslate"><span class="pre">dumps()</span></code></a> function.

  352. Similarly, to de-serialize a data stream, you call the <a class="reference internal" href="#pickle.loads" title="pickle.loads"><code class="xref py py-func docutils literal notranslate"><span class="pre">loads()</span></code></a> function.

  353. However, if you want more control over serialization and de-serialization,

  354. you can create a <a class="reference internal" href="#pickle.Pickler" title="pickle.Pickler"><code class="xref py py-class docutils literal notranslate"><span class="pre">Pickler</span></code></a> or an <a class="reference internal" href="#pickle.Unpickler" title="pickle.Unpickler"><code class="xref py py-class docutils literal notranslate"><span class="pre">Unpickler</span></code></a> object, respectively.</p>

  355. <p>The <a class="reference internal" href="#module-pickle" title="pickle: Convert Python objects to streams of bytes and back."><code class="xref py py-mod docutils literal notranslate"><span class="pre">pickle</span></code></a> module provides the following constants:</p>

  356. <dl class="py data">

  357. <dt class="sig sig-object py" id="pickle.HIGHEST_PROTOCOL">

  358. <span class="sig-prename descclassname"><span class="pre">pickle.</span></span><span class="sig-name descname"><span class="pre">HIGHEST_PROTOCOL</span></span><a class="headerlink" href="#pickle.HIGHEST_PROTOCOL" title="Link to this definition">¶</a></dt>

  359. <dd><p>An integer, the highest <a class="reference internal" href="#pickle-protocols"><span class="std std-ref">protocol version</span></a>

  360. available.  This value can be passed as a <em>protocol</em> value to functions

  361. <a class="reference internal" href="#pickle.dump" title="pickle.dump"><code class="xref py py-func docutils literal notranslate"><span class="pre">dump()</span></code></a> and <a class="reference internal" href="#pickle.dumps" title="pickle.dumps"><code class="xref py py-func docutils literal notranslate"><span class="pre">dumps()</span></code></a> as well as the <a class="reference internal" href="#pickle.Pickler" title="pickle.Pickler"><code class="xref py py-class docutils literal notranslate"><span class="pre">Pickler</span></code></a>

  362. constructor.</p>

  363. </dd></dl>

  364. 

  365. <dl class="py data">

  366. <dt class="sig sig-object py" id="pickle.DEFAULT_PROTOCOL">

  367. <span class="sig-prename descclassname"><span class="pre">pickle.</span></span><span class="sig-name descname"><span class="pre">DEFAULT_PROTOCOL</span></span><a class="headerlink" href="#pickle.DEFAULT_PROTOCOL" title="Link to this definition">¶</a></dt>

  368. <dd><p>An integer, the default <a class="reference internal" href="#pickle-protocols"><span class="std std-ref">protocol version</span></a> used

  369. for pickling.  May be less than <a class="reference internal" href="#pickle.HIGHEST_PROTOCOL" title="pickle.HIGHEST_PROTOCOL"><code class="xref py py-data docutils literal notranslate"><span class="pre">HIGHEST_PROTOCOL</span></code></a>.  Currently the

  370. default protocol is 4, first introduced in Python 3.4 and incompatible

  371. with previous versions.</p>

  372. <div class="versionchanged">

  373. <p><span class="versionmodified changed">Changed in version 3.0: </span>The default protocol is 3.</p>

  374. </div>

  375. <div class="versionchanged">

  376. <p><span class="versionmodified changed">Changed in version 3.8: </span>The default protocol is 4.</p>

  377. </div>

  378. </dd></dl>

  379. 

  380. <p>The <a class="reference internal" href="#module-pickle" title="pickle: Convert Python objects to streams of bytes and back."><code class="xref py py-mod docutils literal notranslate"><span class="pre">pickle</span></code></a> module provides the following functions to make the pickling

  381. process more convenient:</p>

  382. <dl class="py function">

  383. <dt class="sig sig-object py" id="pickle.dump">

  384. <span class="sig-prename descclassname"><span class="pre">pickle.</span></span><span class="sig-name descname"><span class="pre">dump</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">obj</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">file</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">protocol</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">None</span></span></em>, <em class="sig-param"><span class="o"><span class="pre">*</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">fix_imports</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">True</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">buffer_callback</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">None</span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#pickle.dump" title="Link to this definition">¶</a></dt>

  385. <dd><p>Write the pickled representation of the object <em>obj</em> to the open

  386. <a class="reference internal" href="../glossary.html#term-file-object"><span class="xref std std-term">file object</span></a> <em>file</em>.  This is equivalent to

  387. <code class="docutils literal notranslate"><span class="pre">Pickler(file,</span> <span class="pre">protocol).dump(obj)</span></code>.</p>

  388. <p>Arguments <em>file</em>, <em>protocol</em>, <em>fix_imports</em> and <em>buffer_callback</em> have

  389. the same meaning as in the <a class="reference internal" href="#pickle.Pickler" title="pickle.Pickler"><code class="xref py py-class docutils literal notranslate"><span class="pre">Pickler</span></code></a> constructor.</p>

  390. <div class="versionchanged">

  391. <p><span class="versionmodified changed">Changed in version 3.8: </span>The <em>buffer_callback</em> argument was added.</p>

  392. </div>

  393. </dd></dl>

  394. 

  395. <dl class="py function">

  396. <dt class="sig sig-object py" id="pickle.dumps">

  397. <span class="sig-prename descclassname"><span class="pre">pickle.</span></span><span class="sig-name descname"><span class="pre">dumps</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">obj</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">protocol</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">None</span></span></em>, <em class="sig-param"><span class="o"><span class="pre">*</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">fix_imports</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">True</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">buffer_callback</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">None</span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#pickle.dumps" title="Link to this definition">¶</a></dt>

  398. <dd><p>Return the pickled representation of the object <em>obj</em> as a <a class="reference internal" href="stdtypes.html#bytes" title="bytes"><code class="xref py py-class docutils literal notranslate"><span class="pre">bytes</span></code></a> object,

  399. instead of writing it to a file.</p>

  400. <p>Arguments <em>protocol</em>, <em>fix_imports</em> and <em>buffer_callback</em> have the same

  401. meaning as in the <a class="reference internal" href="#pickle.Pickler" title="pickle.Pickler"><code class="xref py py-class docutils literal notranslate"><span class="pre">Pickler</span></code></a> constructor.</p>

  402. <div class="versionchanged">

  403. <p><span class="versionmodified changed">Changed in version 3.8: </span>The <em>buffer_callback</em> argument was added.</p>

  404. </div>

  405. </dd></dl>

  406. 

  407. <dl class="py function">

  408. <dt class="sig sig-object py" id="pickle.load">

  409. <span class="sig-prename descclassname"><span class="pre">pickle.</span></span><span class="sig-name descname"><span class="pre">load</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">file</span></span></em>, <em class="sig-param"><span class="o"><span class="pre">*</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">fix_imports</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">True</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">encoding</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">'ASCII'</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">errors</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">'strict'</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">buffers</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">None</span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#pickle.load" title="Link to this definition">¶</a></dt>

  410. <dd><p>Read the pickled representation of an object from the open <a class="reference internal" href="../glossary.html#term-file-object"><span class="xref std std-term">file object</span></a>

  411. <em>file</em> and return the reconstituted object hierarchy specified therein.

  412. This is equivalent to <code class="docutils literal notranslate"><span class="pre">Unpickler(file).load()</span></code>.</p>

  413. <p>The protocol version of the pickle is detected automatically, so no

  414. protocol argument is needed.  Bytes past the pickled representation

  415. of the object are ignored.</p>

  416. <p>Arguments <em>file</em>, <em>fix_imports</em>, <em>encoding</em>, <em>errors</em>, <em>strict</em> and <em>buffers</em>

  417. have the same meaning as in the <a class="reference internal" href="#pickle.Unpickler" title="pickle.Unpickler"><code class="xref py py-class docutils literal notranslate"><span class="pre">Unpickler</span></code></a> constructor.</p>

  418. <div class="versionchanged">

  419. <p><span class="versionmodified changed">Changed in version 3.8: </span>The <em>buffers</em> argument was added.</p>

  420. </div>

  421. </dd></dl>

  422. 

  423. <dl class="py function">

  424. <dt class="sig sig-object py" id="pickle.loads">

  425. <span class="sig-prename descclassname"><span class="pre">pickle.</span></span><span class="sig-name descname"><span class="pre">loads</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">data</span></span></em>, <em class="sig-param"><span class="o"><span class="pre">/</span></span></em>, <em class="sig-param"><span class="o"><span class="pre">*</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">fix_imports</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">True</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">encoding</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">'ASCII'</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">errors</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">'strict'</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">buffers</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">None</span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#pickle.loads" title="Link to this definition">¶</a></dt>

  426. <dd><p>Return the reconstituted object hierarchy of the pickled representation

  427. <em>data</em> of an object. <em>data</em> must be a <a class="reference internal" href="../glossary.html#term-bytes-like-object"><span class="xref std std-term">bytes-like object</span></a>.</p>

  428. <p>The protocol version of the pickle is detected automatically, so no

  429. protocol argument is needed.  Bytes past the pickled representation

  430. of the object are ignored.</p>

  431. <p>Arguments <em>fix_imports</em>, <em>encoding</em>, <em>errors</em>, <em>strict</em> and <em>buffers</em>

  432. have the same meaning as in the <a class="reference internal" href="#pickle.Unpickler" title="pickle.Unpickler"><code class="xref py py-class docutils literal notranslate"><span class="pre">Unpickler</span></code></a> constructor.</p>

  433. <div class="versionchanged">

  434. <p><span class="versionmodified changed">Changed in version 3.8: </span>The <em>buffers</em> argument was added.</p>

  435. </div>

  436. </dd></dl>

  437. 

  438. <p>The <a class="reference internal" href="#module-pickle" title="pickle: Convert Python objects to streams of bytes and back."><code class="xref py py-mod docutils literal notranslate"><span class="pre">pickle</span></code></a> module defines three exceptions:</p>

  439. <dl class="py exception">

  440. <dt class="sig sig-object py" id="pickle.PickleError">

  441. <em class="property"><span class="pre">exception</span><span class="w"> </span></em><span class="sig-prename descclassname"><span class="pre">pickle.</span></span><span class="sig-name descname"><span class="pre">PickleError</span></span><a class="headerlink" href="#pickle.PickleError" title="Link to this definition">¶</a></dt>

  442. <dd><p>Common base class for the other pickling exceptions.  It inherits from

  443. <a class="reference internal" href="exceptions.html#Exception" title="Exception"><code class="xref py py-exc docutils literal notranslate"><span class="pre">Exception</span></code></a>.</p>

  444. </dd></dl>

  445. 

  446. <dl class="py exception">

  447. <dt class="sig sig-object py" id="pickle.PicklingError">

  448. <em class="property"><span class="pre">exception</span><span class="w"> </span></em><span class="sig-prename descclassname"><span class="pre">pickle.</span></span><span class="sig-name descname"><span class="pre">PicklingError</span></span><a class="headerlink" href="#pickle.PicklingError" title="Link to this definition">¶</a></dt>

  449. <dd><p>Error raised when an unpicklable object is encountered by <a class="reference internal" href="#pickle.Pickler" title="pickle.Pickler"><code class="xref py py-class docutils literal notranslate"><span class="pre">Pickler</span></code></a>.

  450. It inherits from <a class="reference internal" href="#pickle.PickleError" title="pickle.PickleError"><code class="xref py py-exc docutils literal notranslate"><span class="pre">PickleError</span></code></a>.</p>

  451. <p>Refer to <a class="reference internal" href="#pickle-picklable"><span class="std std-ref">What can be pickled and unpickled?</span></a> to learn what kinds of objects can be

  452. pickled.</p>

  453. </dd></dl>

  454. 

  455. <dl class="py exception">

  456. <dt class="sig sig-object py" id="pickle.UnpicklingError">

  457. <em class="property"><span class="pre">exception</span><span class="w"> </span></em><span class="sig-prename descclassname"><span class="pre">pickle.</span></span><span class="sig-name descname"><span class="pre">UnpicklingError</span></span><a class="headerlink" href="#pickle.UnpicklingError" title="Link to this definition">¶</a></dt>

  458. <dd><p>Error raised when there is a problem unpickling an object, such as a data

  459. corruption or a security violation.  It inherits from <a class="reference internal" href="#pickle.PickleError" title="pickle.PickleError"><code class="xref py py-exc docutils literal notranslate"><span class="pre">PickleError</span></code></a>.</p>

  460. <p>Note that other exceptions may also be raised during unpickling, including

  461. (but not necessarily limited to) AttributeError, EOFError, ImportError, and

  462. IndexError.</p>

  463. </dd></dl>

  464. 

  465. <p>The <a class="reference internal" href="#module-pickle" title="pickle: Convert Python objects to streams of bytes and back."><code class="xref py py-mod docutils literal notranslate"><span class="pre">pickle</span></code></a> module exports three classes, <a class="reference internal" href="#pickle.Pickler" title="pickle.Pickler"><code class="xref py py-class docutils literal notranslate"><span class="pre">Pickler</span></code></a>,

  466. <a class="reference internal" href="#pickle.Unpickler" title="pickle.Unpickler"><code class="xref py py-class docutils literal notranslate"><span class="pre">Unpickler</span></code></a> and <a class="reference internal" href="#pickle.PickleBuffer" title="pickle.PickleBuffer"><code class="xref py py-class docutils literal notranslate"><span class="pre">PickleBuffer</span></code></a>:</p>

  467. <dl class="py class">

  468. <dt class="sig sig-object py" id="pickle.Pickler">

  469. <em class="property"><span class="pre">class</span><span class="w"> </span></em><span class="sig-prename descclassname"><span class="pre">pickle.</span></span><span class="sig-name descname"><span class="pre">Pickler</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">file</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">protocol</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">None</span></span></em>, <em class="sig-param"><span class="o"><span class="pre">*</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">fix_imports</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">True</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">buffer_callback</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">None</span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#pickle.Pickler" title="Link to this definition">¶</a></dt>

  470. <dd><p>This takes a binary file for writing a pickle data stream.</p>

  471. <p>The optional <em>protocol</em> argument, an integer, tells the pickler to use

  472. the given protocol; supported protocols are 0 to <a class="reference internal" href="#pickle.HIGHEST_PROTOCOL" title="pickle.HIGHEST_PROTOCOL"><code class="xref py py-data docutils literal notranslate"><span class="pre">HIGHEST_PROTOCOL</span></code></a>.

  473. If not specified, the default is <a class="reference internal" href="#pickle.DEFAULT_PROTOCOL" title="pickle.DEFAULT_PROTOCOL"><code class="xref py py-data docutils literal notranslate"><span class="pre">DEFAULT_PROTOCOL</span></code></a>.  If a negative

  474. number is specified, <a class="reference internal" href="#pickle.HIGHEST_PROTOCOL" title="pickle.HIGHEST_PROTOCOL"><code class="xref py py-data docutils literal notranslate"><span class="pre">HIGHEST_PROTOCOL</span></code></a> is selected.</p>

  475. <p>The <em>file</em> argument must have a write() method that accepts a single bytes

  476. argument.  It can thus be an on-disk file opened for binary writing, an

  477. <a class="reference internal" href="io.html#io.BytesIO" title="io.BytesIO"><code class="xref py py-class docutils literal notranslate"><span class="pre">io.BytesIO</span></code></a> instance, or any other custom object that meets this

  478. interface.</p>

  479. <p>If <em>fix_imports</em> is true and <em>protocol</em> is less than 3, pickle will try to

  480. map the new Python 3 names to the old module names used in Python 2, so

  481. that the pickle data stream is readable with Python 2.</p>

  482. <p>If <em>buffer_callback</em> is None (the default), buffer views are

  483. serialized into <em>file</em> as part of the pickle stream.</p>

  484. <p>If <em>buffer_callback</em> is not None, then it can be called any number

  485. of times with a buffer view.  If the callback returns a false value

  486. (such as None), the given buffer is <a class="reference internal" href="#pickle-oob"><span class="std std-ref">out-of-band</span></a>;

  487. otherwise the buffer is serialized in-band, i.e. inside the pickle stream.</p>

  488. <p>It is an error if <em>buffer_callback</em> is not None and <em>protocol</em> is

  489. None or smaller than 5.</p>

  490. <div class="versionchanged">

  491. <p><span class="versionmodified changed">Changed in version 3.8: </span>The <em>buffer_callback</em> argument was added.</p>

  492. </div>

  493. <dl class="py method">

  494. <dt class="sig sig-object py" id="pickle.Pickler.dump">

  495. <span class="sig-name descname"><span class="pre">dump</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">obj</span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#pickle.Pickler.dump" title="Link to this definition">¶</a></dt>

  496. <dd><p>Write the pickled representation of <em>obj</em> to the open file object given in

  497. the constructor.</p>

  498. </dd></dl>

  499. 

  500. <dl class="py method">

  501. <dt class="sig sig-object py" id="pickle.Pickler.persistent_id">

  502. <span class="sig-name descname"><span class="pre">persistent_id</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">obj</span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#pickle.Pickler.persistent_id" title="Link to this definition">¶</a></dt>

  503. <dd><p>Do nothing by default.  This exists so a subclass can override it.</p>

  504. <p>If <a class="reference internal" href="#pickle.Pickler.persistent_id" title="pickle.Pickler.persistent_id"><code class="xref py py-meth docutils literal notranslate"><span class="pre">persistent_id()</span></code></a> returns <code class="docutils literal notranslate"><span class="pre">None</span></code>, <em>obj</em> is pickled as usual.  Any

  505. other value causes <a class="reference internal" href="#pickle.Pickler" title="pickle.Pickler"><code class="xref py py-class docutils literal notranslate"><span class="pre">Pickler</span></code></a> to emit the returned value as a

  506. persistent ID for <em>obj</em>.  The meaning of this persistent ID should be

  507. defined by <a class="reference internal" href="#pickle.Unpickler.persistent_load" title="pickle.Unpickler.persistent_load"><code class="xref py py-meth docutils literal notranslate"><span class="pre">Unpickler.persistent_load()</span></code></a>.  Note that the value

  508. returned by <a class="reference internal" href="#pickle.Pickler.persistent_id" title="pickle.Pickler.persistent_id"><code class="xref py py-meth docutils literal notranslate"><span class="pre">persistent_id()</span></code></a> cannot itself have a persistent ID.</p>

  509. <p>See <a class="reference internal" href="#pickle-persistent"><span class="std std-ref">Persistence of External Objects</span></a> for details and examples of uses.</p>

  510. </dd></dl>

  511. 

  512. <dl class="py attribute">

  513. <dt class="sig sig-object py" id="pickle.Pickler.dispatch_table">

  514. <span class="sig-name descname"><span class="pre">dispatch_table</span></span><a class="headerlink" href="#pickle.Pickler.dispatch_table" title="Link to this definition">¶</a></dt>

  515. <dd><p>A pickler object’s dispatch table is a registry of <em>reduction

  516. functions</em> of the kind which can be declared using

  517. <a class="reference internal" href="copyreg.html#copyreg.pickle" title="copyreg.pickle"><code class="xref py py-func docutils literal notranslate"><span class="pre">copyreg.pickle()</span></code></a>.  It is a mapping whose keys are classes

  518. and whose values are reduction functions.  A reduction function

  519. takes a single argument of the associated class and should

  520. conform to the same interface as a <a class="reference internal" href="#object.__reduce__" title="object.__reduce__"><code class="xref py py-meth docutils literal notranslate"><span class="pre">__reduce__()</span></code></a>

  521. method.</p>

  522. <p>By default, a pickler object will not have a

  523. <a class="reference internal" href="#pickle.Pickler.dispatch_table" title="pickle.Pickler.dispatch_table"><code class="xref py py-attr docutils literal notranslate"><span class="pre">dispatch_table</span></code></a> attribute, and it will instead use the

  524. global dispatch table managed by the <a class="reference internal" href="copyreg.html#module-copyreg" title="copyreg: Register pickle support functions."><code class="xref py py-mod docutils literal notranslate"><span class="pre">copyreg</span></code></a> module.

  525. However, to customize the pickling for a specific pickler object

  526. one can set the <a class="reference internal" href="#pickle.Pickler.dispatch_table" title="pickle.Pickler.dispatch_table"><code class="xref py py-attr docutils literal notranslate"><span class="pre">dispatch_table</span></code></a> attribute to a dict-like

  527. object.  Alternatively, if a subclass of <a class="reference internal" href="#pickle.Pickler" title="pickle.Pickler"><code class="xref py py-class docutils literal notranslate"><span class="pre">Pickler</span></code></a> has a

  528. <a class="reference internal" href="#pickle.Pickler.dispatch_table" title="pickle.Pickler.dispatch_table"><code class="xref py py-attr docutils literal notranslate"><span class="pre">dispatch_table</span></code></a> attribute then this will be used as the

  529. default dispatch table for instances of that class.</p>

  530. <p>See <a class="reference internal" href="#pickle-dispatch"><span class="std std-ref">Dispatch Tables</span></a> for usage examples.</p>

  531. <div class="versionadded">

  532. <p><span class="versionmodified added">New in version 3.3.</span></p>

  533. </div>

  534. </dd></dl>

  535. 

  536. <dl class="py method">

  537. <dt class="sig sig-object py" id="pickle.Pickler.reducer_override">

  538. <span class="sig-name descname"><span class="pre">reducer_override</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">obj</span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#pickle.Pickler.reducer_override" title="Link to this definition">¶</a></dt>

  539. <dd><p>Special reducer that can be defined in <a class="reference internal" href="#pickle.Pickler" title="pickle.Pickler"><code class="xref py py-class docutils literal notranslate"><span class="pre">Pickler</span></code></a> subclasses. This

  540. method has priority over any reducer in the <a class="reference internal" href="#pickle.Pickler.dispatch_table" title="pickle.Pickler.dispatch_table"><code class="xref py py-attr docutils literal notranslate"><span class="pre">dispatch_table</span></code></a>.  It

  541. should conform to the same interface as a <a class="reference internal" href="#object.__reduce__" title="object.__reduce__"><code class="xref py py-meth docutils literal notranslate"><span class="pre">__reduce__()</span></code></a> method, and

  542. can optionally return <a class="reference internal" href="constants.html#NotImplemented" title="NotImplemented"><code class="xref py py-data docutils literal notranslate"><span class="pre">NotImplemented</span></code></a> to fallback on

  543. <a class="reference internal" href="#pickle.Pickler.dispatch_table" title="pickle.Pickler.dispatch_table"><code class="xref py py-attr docutils literal notranslate"><span class="pre">dispatch_table</span></code></a>-registered reducers to pickle <code class="docutils literal notranslate"><span class="pre">obj</span></code>.</p>

  544. <p>For a detailed example, see <a class="reference internal" href="#reducer-override"><span class="std std-ref">Custom Reduction for Types, Functions, and Other Objects</span></a>.</p>

  545. <div class="versionadded">

  546. <p><span class="versionmodified added">New in version 3.8.</span></p>

  547. </div>

  548. </dd></dl>

  549. 

  550. <dl class="py attribute">

  551. <dt class="sig sig-object py" id="pickle.Pickler.fast">

  552. <span class="sig-name descname"><span class="pre">fast</span></span><a class="headerlink" href="#pickle.Pickler.fast" title="Link to this definition">¶</a></dt>

  553. <dd><p>Deprecated. Enable fast mode if set to a true value.  The fast mode

  554. disables the usage of memo, therefore speeding the pickling process by not

  555. generating superfluous PUT opcodes.  It should not be used with

  556. self-referential objects, doing otherwise will cause <a class="reference internal" href="#pickle.Pickler" title="pickle.Pickler"><code class="xref py py-class docutils literal notranslate"><span class="pre">Pickler</span></code></a> to

  557. recurse infinitely.</p>

  558. <p>Use <a class="reference internal" href="pickletools.html#pickletools.optimize" title="pickletools.optimize"><code class="xref py py-func docutils literal notranslate"><span class="pre">pickletools.optimize()</span></code></a> if you need more compact pickles.</p>

  559. </dd></dl>

  560. 

  561. </dd></dl>

  562. 

  563. <dl class="py class">

  564. <dt class="sig sig-object py" id="pickle.Unpickler">

  565. <em class="property"><span class="pre">class</span><span class="w"> </span></em><span class="sig-prename descclassname"><span class="pre">pickle.</span></span><span class="sig-name descname"><span class="pre">Unpickler</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">file</span></span></em>, <em class="sig-param"><span class="o"><span class="pre">*</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">fix_imports</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">True</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">encoding</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">'ASCII'</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">errors</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">'strict'</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">buffers</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">None</span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#pickle.Unpickler" title="Link to this definition">¶</a></dt>

  566. <dd><p>This takes a binary file for reading a pickle data stream.</p>

  567. <p>The protocol version of the pickle is detected automatically, so no

  568. protocol argument is needed.</p>

  569. <p>The argument <em>file</em> must have three methods, a read() method that takes an

  570. integer argument, a readinto() method that takes a buffer argument

  571. and a readline() method that requires no arguments, as in the

  572. <a class="reference internal" href="io.html#io.BufferedIOBase" title="io.BufferedIOBase"><code class="xref py py-class docutils literal notranslate"><span class="pre">io.BufferedIOBase</span></code></a> interface.  Thus <em>file</em> can be an on-disk file

  573. opened for binary reading, an <a class="reference internal" href="io.html#io.BytesIO" title="io.BytesIO"><code class="xref py py-class docutils literal notranslate"><span class="pre">io.BytesIO</span></code></a> object, or any other

  574. custom object that meets this interface.</p>

  575. <p>The optional arguments <em>fix_imports</em>, <em>encoding</em> and <em>errors</em> are used

  576. to control compatibility support for pickle stream generated by Python 2.

  577. If <em>fix_imports</em> is true, pickle will try to map the old Python 2 names

  578. to the new names used in Python 3.  The <em>encoding</em> and <em>errors</em> tell

  579. pickle how to decode 8-bit string instances pickled by Python 2;

  580. these default to ‘ASCII’ and ‘strict’, respectively.  The <em>encoding</em> can

  581. be ‘bytes’ to read these 8-bit string instances as bytes objects.

  582. Using <code class="docutils literal notranslate"><span class="pre">encoding='latin1'</span></code> is required for unpickling NumPy arrays and

  583. instances of <a class="reference internal" href="datetime.html#datetime.datetime" title="datetime.datetime"><code class="xref py py-class docutils literal notranslate"><span class="pre">datetime</span></code></a>, <a class="reference internal" href="datetime.html#datetime.date" title="datetime.date"><code class="xref py py-class docutils literal notranslate"><span class="pre">date</span></code></a> and

  584. <a class="reference internal" href="datetime.html#datetime.time" title="datetime.time"><code class="xref py py-class docutils literal notranslate"><span class="pre">time</span></code></a> pickled by Python 2.</p>

  585. <p>If <em>buffers</em> is None (the default), then all data necessary for

  586. deserialization must be contained in the pickle stream.  This means

  587. that the <em>buffer_callback</em> argument was None when a <a class="reference internal" href="#pickle.Pickler" title="pickle.Pickler"><code class="xref py py-class docutils literal notranslate"><span class="pre">Pickler</span></code></a>

  588. was instantiated (or when <a class="reference internal" href="#pickle.dump" title="pickle.dump"><code class="xref py py-func docutils literal notranslate"><span class="pre">dump()</span></code></a> or <a class="reference internal" href="#pickle.dumps" title="pickle.dumps"><code class="xref py py-func docutils literal notranslate"><span class="pre">dumps()</span></code></a> was called).</p>

  589. <p>If <em>buffers</em> is not None, it should be an iterable of buffer-enabled

  590. objects that is consumed each time the pickle stream references

  591. an <a class="reference internal" href="#pickle-oob"><span class="std std-ref">out-of-band</span></a> buffer view.  Such buffers have been

  592. given in order to the <em>buffer_callback</em> of a Pickler object.</p>

  593. <div class="versionchanged">

  594. <p><span class="versionmodified changed">Changed in version 3.8: </span>The <em>buffers</em> argument was added.</p>

  595. </div>

  596. <dl class="py method">

  597. <dt class="sig sig-object py" id="pickle.Unpickler.load">

  598. <span class="sig-name descname"><span class="pre">load</span></span><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="headerlink" href="#pickle.Unpickler.load" title="Link to this definition">¶</a></dt>

  599. <dd><p>Read the pickled representation of an object from the open file object

  600. given in the constructor, and return the reconstituted object hierarchy

  601. specified therein.  Bytes past the pickled representation of the object

  602. are ignored.</p>

  603. </dd></dl>

  604. 

  605. <dl class="py method">

  606. <dt class="sig sig-object py" id="pickle.Unpickler.persistent_load">

  607. <span class="sig-name descname"><span class="pre">persistent_load</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">pid</span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#pickle.Unpickler.persistent_load" title="Link to this definition">¶</a></dt>

  608. <dd><p>Raise an <a class="reference internal" href="#pickle.UnpicklingError" title="pickle.UnpicklingError"><code class="xref py py-exc docutils literal notranslate"><span class="pre">UnpicklingError</span></code></a> by default.</p>

  609. <p>If defined, <a class="reference internal" href="#pickle.Unpickler.persistent_load" title="pickle.Unpickler.persistent_load"><code class="xref py py-meth docutils literal notranslate"><span class="pre">persistent_load()</span></code></a> should return the object specified by

  610. the persistent ID <em>pid</em>.  If an invalid persistent ID is encountered, an

  611. <a class="reference internal" href="#pickle.UnpicklingError" title="pickle.UnpicklingError"><code class="xref py py-exc docutils literal notranslate"><span class="pre">UnpicklingError</span></code></a> should be raised.</p>

  612. <p>See <a class="reference internal" href="#pickle-persistent"><span class="std std-ref">Persistence of External Objects</span></a> for details and examples of uses.</p>

  613. </dd></dl>

  614. 

  615. <dl class="py method">

  616. <dt class="sig sig-object py" id="pickle.Unpickler.find_class">

  617. <span class="sig-name descname"><span class="pre">find_class</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">module</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">name</span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#pickle.Unpickler.find_class" title="Link to this definition">¶</a></dt>

  618. <dd><p>Import <em>module</em> if necessary and return the object called <em>name</em> from it,

  619. where the <em>module</em> and <em>name</em> arguments are <a class="reference internal" href="stdtypes.html#str" title="str"><code class="xref py py-class docutils literal notranslate"><span class="pre">str</span></code></a> objects.  Note,

  620. unlike its name suggests, <a class="reference internal" href="#pickle.Unpickler.find_class" title="pickle.Unpickler.find_class"><code class="xref py py-meth docutils literal notranslate"><span class="pre">find_class()</span></code></a> is also used for finding

  621. functions.</p>

  622. <p>Subclasses may override this to gain control over what type of objects and

  623. how they can be loaded, potentially reducing security risks. Refer to

  624. <a class="reference internal" href="#pickle-restrict"><span class="std std-ref">Restricting Globals</span></a> for details.</p>

  625. <p class="audit-hook">Raises an <a class="reference internal" href="sys.html#auditing"><span class="std std-ref">auditing event</span></a> <code class="docutils literal notranslate"><span class="pre">pickle.find_class</span></code> with arguments <code class="docutils literal notranslate"><span class="pre">module</span></code>, <code class="docutils literal notranslate"><span class="pre">name</span></code>.</p>

  626. </dd></dl>

  627. 

  628. </dd></dl>

  629. 

  630. <dl class="py class">

  631. <dt class="sig sig-object py" id="pickle.PickleBuffer">

  632. <em class="property"><span class="pre">class</span><span class="w"> </span></em><span class="sig-prename descclassname"><span class="pre">pickle.</span></span><span class="sig-name descname"><span class="pre">PickleBuffer</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">buffer</span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#pickle.PickleBuffer" title="Link to this definition">¶</a></dt>

  633. <dd><p>A wrapper for a buffer representing picklable data.  <em>buffer</em> must be a

  634. <a class="reference internal" href="../c-api/buffer.html#bufferobjects"><span class="std std-ref">buffer-providing</span></a> object, such as a

  635. <a class="reference internal" href="../glossary.html#term-bytes-like-object"><span class="xref std std-term">bytes-like object</span></a> or a N-dimensional array.</p>

  636. <p><a class="reference internal" href="#pickle.PickleBuffer" title="pickle.PickleBuffer"><code class="xref py py-class docutils literal notranslate"><span class="pre">PickleBuffer</span></code></a> is itself a buffer provider, therefore it is

  637. possible to pass it to other APIs expecting a buffer-providing object,

  638. such as <a class="reference internal" href="stdtypes.html#memoryview" title="memoryview"><code class="xref py py-class docutils literal notranslate"><span class="pre">memoryview</span></code></a>.</p>

  639. <p><a class="reference internal" href="#pickle.PickleBuffer" title="pickle.PickleBuffer"><code class="xref py py-class docutils literal notranslate"><span class="pre">PickleBuffer</span></code></a> objects can only be serialized using pickle

  640. protocol 5 or higher.  They are eligible for

  641. <a class="reference internal" href="#pickle-oob"><span class="std std-ref">out-of-band serialization</span></a>.</p>

  642. <div class="versionadded">

  643. <p><span class="versionmodified added">New in version 3.8.</span></p>

  644. </div>

  645. <dl class="py method">

  646. <dt class="sig sig-object py" id="pickle.PickleBuffer.raw">

  647. <span class="sig-name descname"><span class="pre">raw</span></span><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="headerlink" href="#pickle.PickleBuffer.raw" title="Link to this definition">¶</a></dt>

  648. <dd><p>Return a <a class="reference internal" href="stdtypes.html#memoryview" title="memoryview"><code class="xref py py-class docutils literal notranslate"><span class="pre">memoryview</span></code></a> of the memory area underlying this buffer.

  649. The returned object is a one-dimensional, C-contiguous memoryview

  650. with format <code class="docutils literal notranslate"><span class="pre">B</span></code> (unsigned bytes).  <a class="reference internal" href="exceptions.html#BufferError" title="BufferError"><code class="xref py py-exc docutils literal notranslate"><span class="pre">BufferError</span></code></a> is raised if

  651. the buffer is neither C- nor Fortran-contiguous.</p>

  652. </dd></dl>

  653. 

  654. <dl class="py method">

  655. <dt class="sig sig-object py" id="pickle.PickleBuffer.release">

  656. <span class="sig-name descname"><span class="pre">release</span></span><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="headerlink" href="#pickle.PickleBuffer.release" title="Link to this definition">¶</a></dt>

  657. <dd><p>Release the underlying buffer exposed by the PickleBuffer object.</p>

  658. </dd></dl>

  659. 

  660. </dd></dl>

  661. 

  662. </section>

  663. <section id="what-can-be-pickled-and-unpickled">

  664. <span id="pickle-picklable"></span><h2>What can be pickled and unpickled?<a class="headerlink" href="#what-can-be-pickled-and-unpickled" title="Link to this heading">¶</a></h2>

  665. <p>The following types can be pickled:</p>

  666. <ul class="simple">

  667. <li><p>built-in constants (<code class="docutils literal notranslate"><span class="pre">None</span></code>, <code class="docutils literal notranslate"><span class="pre">True</span></code>, <code class="docutils literal notranslate"><span class="pre">False</span></code>, <code class="docutils literal notranslate"><span class="pre">Ellipsis</span></code>, and

  668. <a class="reference internal" href="constants.html#NotImplemented" title="NotImplemented"><code class="xref py py-data docutils literal notranslate"><span class="pre">NotImplemented</span></code></a>);</p></li>

  669. <li><p>integers, floating-point numbers, complex numbers;</p></li>

  670. <li><p>strings, bytes, bytearrays;</p></li>

  671. <li><p>tuples, lists, sets, and dictionaries containing only picklable objects;</p></li>

  672. <li><p>functions (built-in and user-defined) accessible from the top level of a

  673. module (using <a class="reference internal" href="../reference/compound_stmts.html#def"><code class="xref std std-keyword docutils literal notranslate"><span class="pre">def</span></code></a>, not <a class="reference internal" href="../reference/expressions.html#lambda"><code class="xref std std-keyword docutils literal notranslate"><span class="pre">lambda</span></code></a>);</p></li>

  674. <li><p>classes accessible from the top level of a module;</p></li>

  675. <li><p>instances of such classes whose the result of calling <a class="reference internal" href="#object.__getstate__" title="object.__getstate__"><code class="xref py py-meth docutils literal notranslate"><span class="pre">__getstate__()</span></code></a>

  676. is picklable  (see section <a class="reference internal" href="#pickle-inst"><span class="std std-ref">Pickling Class Instances</span></a> for details).</p></li>

  677. </ul>

  678. <p>Attempts to pickle unpicklable objects will raise the <a class="reference internal" href="#pickle.PicklingError" title="pickle.PicklingError"><code class="xref py py-exc docutils literal notranslate"><span class="pre">PicklingError</span></code></a>

  679. exception; when this happens, an unspecified number of bytes may have already

  680. been written to the underlying file.  Trying to pickle a highly recursive data

  681. structure may exceed the maximum recursion depth, a <a class="reference internal" href="exceptions.html#RecursionError" title="RecursionError"><code class="xref py py-exc docutils literal notranslate"><span class="pre">RecursionError</span></code></a> will be

  682. raised in this case.  You can carefully raise this limit with

  683. <a class="reference internal" href="sys.html#sys.setrecursionlimit" title="sys.setrecursionlimit"><code class="xref py py-func docutils literal notranslate"><span class="pre">sys.setrecursionlimit()</span></code></a>.</p>

  684. <p>Note that functions (built-in and user-defined) are pickled by fully

  685. <a class="reference internal" href="../glossary.html#term-qualified-name"><span class="xref std std-term">qualified name</span></a>, not by value. <a class="footnote-reference brackets" href="#id8" id="id3" role="doc-noteref"><span class="fn-bracket">[</span>2<span class="fn-bracket">]</span></a>  This means that only the function name is

  686. pickled, along with the name of the containing module and classes.  Neither

  687. the function’s code, nor any of its function attributes are pickled.  Thus the

  688. defining module must be importable in the unpickling environment, and the module

  689. must contain the named object, otherwise an exception will be raised. <a class="footnote-reference brackets" href="#id9" id="id4" role="doc-noteref"><span class="fn-bracket">[</span>3<span class="fn-bracket">]</span></a></p>

  690. <p>Similarly, classes are pickled by fully qualified name, so the same restrictions in

  691. the unpickling environment apply.  Note that none of the class’s code or data is

  692. pickled, so in the following example the class attribute <code class="docutils literal notranslate"><span class="pre">attr</span></code> is not

  693. restored in the unpickling environment:</p>

  694. <div class="highlight-python3 notranslate"><div class="highlight"><pre><span></span><span class="k">class</span> <span class="nc">Foo</span><span class="p">:</span>

  695.     <span class="n">attr</span> <span class="o">=</span> <span class="s1">&#39;A class attribute&#39;</span>

  696. 

  697. <span class="n">picklestring</span> <span class="o">=</span> <span class="n">pickle</span><span class="o">.</span><span class="n">dumps</span><span class="p">(</span><span class="n">Foo</span><span class="p">)</span>

  698. </pre></div>

  699. </div>

  700. <p>These restrictions are why picklable functions and classes must be defined at

  701. the top level of a module.</p>

  702. <p>Similarly, when class instances are pickled, their class’s code and data are not

  703. pickled along with them.  Only the instance data are pickled.  This is done on

  704. purpose, so you can fix bugs in a class or add methods to the class and still

  705. load objects that were created with an earlier version of the class.  If you

  706. plan to have long-lived objects that will see many versions of a class, it may

  707. be worthwhile to put a version number in the objects so that suitable

  708. conversions can be made by the class’s <a class="reference internal" href="#object.__setstate__" title="object.__setstate__"><code class="xref py py-meth docutils literal notranslate"><span class="pre">__setstate__()</span></code></a> method.</p>

  709. </section>

  710. <section id="pickling-class-instances">

  711. <span id="pickle-inst"></span><h2>Pickling Class Instances<a class="headerlink" href="#pickling-class-instances" title="Link to this heading">¶</a></h2>

  712. <p>In this section, we describe the general mechanisms available to you to define,

  713. customize, and control how class instances are pickled and unpickled.</p>

  714. <p>In most cases, no additional code is needed to make instances picklable.  By

  715. default, pickle will retrieve the class and the attributes of an instance via

  716. introspection. When a class instance is unpickled, its <a class="reference internal" href="../reference/datamodel.html#object.__init__" title="object.__init__"><code class="xref py py-meth docutils literal notranslate"><span class="pre">__init__()</span></code></a> method

  717. is usually <em>not</em> invoked.  The default behaviour first creates an uninitialized

  718. instance and then restores the saved attributes.  The following code shows an

  719. implementation of this behaviour:</p>

  720. <div class="highlight-python3 notranslate"><div class="highlight"><pre><span></span><span class="k">def</span> <span class="nf">save</span><span class="p">(</span><span class="n">obj</span><span class="p">):</span>

  721.     <span class="k">return</span> <span class="p">(</span><span class="n">obj</span><span class="o">.</span><span class="vm">__class__</span><span class="p">,</span> <span class="n">obj</span><span class="o">.</span><span class="vm">__dict__</span><span class="p">)</span>

  722. 

  723. <span class="k">def</span> <span class="nf">restore</span><span class="p">(</span><span class="bp">cls</span><span class="p">,</span> <span class="n">attributes</span><span class="p">):</span>

  724.     <span class="n">obj</span> <span class="o">=</span> <span class="bp">cls</span><span class="o">.</span><span class="fm">__new__</span><span class="p">(</span><span class="bp">cls</span><span class="p">)</span>

  725.     <span class="n">obj</span><span class="o">.</span><span class="vm">__dict__</span><span class="o">.</span><span class="n">update</span><span class="p">(</span><span class="n">attributes</span><span class="p">)</span>

  726.     <span class="k">return</span> <span class="n">obj</span>

  727. </pre></div>

  728. </div>

  729. <p>Classes can alter the default behaviour by providing one or several special

  730. methods:</p>

  731. <dl class="py method">

  732. <dt class="sig sig-object py" id="object.__getnewargs_ex__">

  733. <span class="sig-prename descclassname"><span class="pre">object.</span></span><span class="sig-name descname"><span class="pre">__getnewargs_ex__</span></span><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="headerlink" href="#object.__getnewargs_ex__" title="Link to this definition">¶</a></dt>

  734. <dd><p>In protocols 2 and newer, classes that implements the

  735. <a class="reference internal" href="#object.__getnewargs_ex__" title="object.__getnewargs_ex__"><code class="xref py py-meth docutils literal notranslate"><span class="pre">__getnewargs_ex__()</span></code></a> method can dictate the values passed to the

  736. <a class="reference internal" href="../reference/datamodel.html#object.__new__" title="object.__new__"><code class="xref py py-meth docutils literal notranslate"><span class="pre">__new__()</span></code></a> method upon unpickling.  The method must return a pair

  737. <code class="docutils literal notranslate"><span class="pre">(args,</span> <span class="pre">kwargs)</span></code> where <em>args</em> is a tuple of positional arguments

  738. and <em>kwargs</em> a dictionary of named arguments for constructing the

  739. object.  Those will be passed to the <a class="reference internal" href="../reference/datamodel.html#object.__new__" title="object.__new__"><code class="xref py py-meth docutils literal notranslate"><span class="pre">__new__()</span></code></a> method upon

  740. unpickling.</p>

  741. <p>You should implement this method if the <a class="reference internal" href="../reference/datamodel.html#object.__new__" title="object.__new__"><code class="xref py py-meth docutils literal notranslate"><span class="pre">__new__()</span></code></a> method of your

  742. class requires keyword-only arguments.  Otherwise, it is recommended for

  743. compatibility to implement <a class="reference internal" href="#object.__getnewargs__" title="object.__getnewargs__"><code class="xref py py-meth docutils literal notranslate"><span class="pre">__getnewargs__()</span></code></a>.</p>

  744. <div class="versionchanged">

  745. <p><span class="versionmodified changed">Changed in version 3.6: </span><a class="reference internal" href="#object.__getnewargs_ex__" title="object.__getnewargs_ex__"><code class="xref py py-meth docutils literal notranslate"><span class="pre">__getnewargs_ex__()</span></code></a> is now used in protocols 2 and 3.</p>

  746. </div>

  747. </dd></dl>

  748. 

  749. <dl class="py method">

  750. <dt class="sig sig-object py" id="object.__getnewargs__">

  751. <span class="sig-prename descclassname"><span class="pre">object.</span></span><span class="sig-name descname"><span class="pre">__getnewargs__</span></span><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="headerlink" href="#object.__getnewargs__" title="Link to this definition">¶</a></dt>

  752. <dd><p>This method serves a similar purpose as <a class="reference internal" href="#object.__getnewargs_ex__" title="object.__getnewargs_ex__"><code class="xref py py-meth docutils literal notranslate"><span class="pre">__getnewargs_ex__()</span></code></a>, but

  753. supports only positional arguments.  It must return a tuple of arguments

  754. <code class="docutils literal notranslate"><span class="pre">args</span></code> which will be passed to the <a class="reference internal" href="../reference/datamodel.html#object.__new__" title="object.__new__"><code class="xref py py-meth docutils literal notranslate"><span class="pre">__new__()</span></code></a> method upon unpickling.</p>

  755. <p><a class="reference internal" href="#object.__getnewargs__" title="object.__getnewargs__"><code class="xref py py-meth docutils literal notranslate"><span class="pre">__getnewargs__()</span></code></a> will not be called if <a class="reference internal" href="#object.__getnewargs_ex__" title="object.__getnewargs_ex__"><code class="xref py py-meth docutils literal notranslate"><span class="pre">__getnewargs_ex__()</span></code></a> is

  756. defined.</p>

  757. <div class="versionchanged">

  758. <p><span class="versionmodified changed">Changed in version 3.6: </span>Before Python 3.6, <a class="reference internal" href="#object.__getnewargs__" title="object.__getnewargs__"><code class="xref py py-meth docutils literal notranslate"><span class="pre">__getnewargs__()</span></code></a> was called instead of

  759. <a class="reference internal" href="#object.__getnewargs_ex__" title="object.__getnewargs_ex__"><code class="xref py py-meth docutils literal notranslate"><span class="pre">__getnewargs_ex__()</span></code></a> in protocols 2 and 3.</p>

  760. </div>

  761. </dd></dl>

  762. 

  763. <dl class="py method">

  764. <dt class="sig sig-object py" id="object.__getstate__">

  765. <span class="sig-prename descclassname"><span class="pre">object.</span></span><span class="sig-name descname"><span class="pre">__getstate__</span></span><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="headerlink" href="#object.__getstate__" title="Link to this definition">¶</a></dt>

  766. <dd><p>Classes can further influence how their instances are pickled by overriding

  767. the method <a class="reference internal" href="#object.__getstate__" title="object.__getstate__"><code class="xref py py-meth docutils literal notranslate"><span class="pre">__getstate__()</span></code></a>.  It is called and the returned object

  768. is pickled as the contents for the instance, instead of a default state.

  769. There are several cases:</p>

  770. <ul class="simple">

  771. <li><p>For a class that has no instance <a class="reference internal" href="stdtypes.html#object.__dict__" title="object.__dict__"><code class="xref py py-attr docutils literal notranslate"><span class="pre">__dict__</span></code></a> and no

  772. <a class="reference internal" href="../reference/datamodel.html#object.__slots__" title="object.__slots__"><code class="xref py py-attr docutils literal notranslate"><span class="pre">__slots__</span></code></a>, the default state is <code class="docutils literal notranslate"><span class="pre">None</span></code>.</p></li>

  773. <li><p>For a class that has an instance <a class="reference internal" href="stdtypes.html#object.__dict__" title="object.__dict__"><code class="xref py py-attr docutils literal notranslate"><span class="pre">__dict__</span></code></a> and no

  774. <a class="reference internal" href="../reference/datamodel.html#object.__slots__" title="object.__slots__"><code class="xref py py-attr docutils literal notranslate"><span class="pre">__slots__</span></code></a>, the default state is <code class="docutils literal notranslate"><span class="pre">self.__dict__</span></code>.</p></li>

  775. <li><p>For a class that has an instance <a class="reference internal" href="stdtypes.html#object.__dict__" title="object.__dict__"><code class="xref py py-attr docutils literal notranslate"><span class="pre">__dict__</span></code></a> and

  776. <a class="reference internal" href="../reference/datamodel.html#object.__slots__" title="object.__slots__"><code class="xref py py-attr docutils literal notranslate"><span class="pre">__slots__</span></code></a>, the default state is a tuple consisting of two

  777. dictionaries:  <code class="docutils literal notranslate"><span class="pre">self.__dict__</span></code>, and a dictionary mapping slot

  778. names to slot values.  Only slots that have a value are

  779. included in the latter.</p></li>

  780. <li><p>For a class that has <a class="reference internal" href="../reference/datamodel.html#object.__slots__" title="object.__slots__"><code class="xref py py-attr docutils literal notranslate"><span class="pre">__slots__</span></code></a> and no instance

  781. <a class="reference internal" href="stdtypes.html#object.__dict__" title="object.__dict__"><code class="xref py py-attr docutils literal notranslate"><span class="pre">__dict__</span></code></a>, the default state is a tuple whose first item

  782. is <code class="docutils literal notranslate"><span class="pre">None</span></code> and whose second item is a dictionary mapping slot names

  783. to slot values described in the previous bullet.</p></li>

  784. </ul>

  785. <div class="versionchanged">

  786. <p><span class="versionmodified changed">Changed in version 3.11: </span>Added the default implementation of the <code class="docutils literal notranslate"><span class="pre">__getstate__()</span></code> method in the

  787. <a class="reference internal" href="functions.html#object" title="object"><code class="xref py py-class docutils literal notranslate"><span class="pre">object</span></code></a> class.</p>

  788. </div>

  789. </dd></dl>

  790. 

  791. <dl class="py method">

  792. <dt class="sig sig-object py" id="object.__setstate__">

  793. <span class="sig-prename descclassname"><span class="pre">object.</span></span><span class="sig-name descname"><span class="pre">__setstate__</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">state</span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#object.__setstate__" title="Link to this definition">¶</a></dt>

  794. <dd><p>Upon unpickling, if the class defines <a class="reference internal" href="#object.__setstate__" title="object.__setstate__"><code class="xref py py-meth docutils literal notranslate"><span class="pre">__setstate__()</span></code></a>, it is called with

  795. the unpickled state.  In that case, there is no requirement for the state

  796. object to be a dictionary.  Otherwise, the pickled state must be a dictionary

  797. and its items are assigned to the new instance’s dictionary.</p>

  798. <div class="admonition note">

  799. <p class="admonition-title">Note</p>

  800. <p>If <a class="reference internal" href="#object.__reduce__" title="object.__reduce__"><code class="xref py py-meth docutils literal notranslate"><span class="pre">__reduce__()</span></code></a> returns a state with value <code class="docutils literal notranslate"><span class="pre">None</span></code> at pickling,

  801. the <a class="reference internal" href="#object.__setstate__" title="object.__setstate__"><code class="xref py py-meth docutils literal notranslate"><span class="pre">__setstate__()</span></code></a> method will not be called upon unpickling.</p>

  802. </div>

  803. </dd></dl>

  804. 

  805. <p>Refer to the section <a class="reference internal" href="#pickle-state"><span class="std std-ref">Handling Stateful Objects</span></a> for more information about how to use

  806. the methods <a class="reference internal" href="#object.__getstate__" title="object.__getstate__"><code class="xref py py-meth docutils literal notranslate"><span class="pre">__getstate__()</span></code></a> and <a class="reference internal" href="#object.__setstate__" title="object.__setstate__"><code class="xref py py-meth docutils literal notranslate"><span class="pre">__setstate__()</span></code></a>.</p>

  807. <div class="admonition note">

  808. <p class="admonition-title">Note</p>

  809. <p>At unpickling time, some methods like <a class="reference internal" href="../reference/datamodel.html#object.__getattr__" title="object.__getattr__"><code class="xref py py-meth docutils literal notranslate"><span class="pre">__getattr__()</span></code></a>,

  810. <a class="reference internal" href="../reference/datamodel.html#object.__getattribute__" title="object.__getattribute__"><code class="xref py py-meth docutils literal notranslate"><span class="pre">__getattribute__()</span></code></a>, or <a class="reference internal" href="../reference/datamodel.html#object.__setattr__" title="object.__setattr__"><code class="xref py py-meth docutils literal notranslate"><span class="pre">__setattr__()</span></code></a> may be called upon the

  811. instance.  In case those methods rely on some internal invariant being

  812. true, the type should implement <a class="reference internal" href="../reference/datamodel.html#object.__new__" title="object.__new__"><code class="xref py py-meth docutils literal notranslate"><span class="pre">__new__()</span></code></a> to establish such an

  813. invariant, as <a class="reference internal" href="../reference/datamodel.html#object.__init__" title="object.__init__"><code class="xref py py-meth docutils literal notranslate"><span class="pre">__init__()</span></code></a> is not called when unpickling an

  814. instance.</p>

  815. </div>

  816. <p id="index-5">As we shall see, pickle does not use directly the methods described above.  In

  817. fact, these methods are part of the copy protocol which implements the

  818. <a class="reference internal" href="#object.__reduce__" title="object.__reduce__"><code class="xref py py-meth docutils literal notranslate"><span class="pre">__reduce__()</span></code></a> special method.  The copy protocol provides a unified

  819. interface for retrieving the data necessary for pickling and copying

  820. objects. <a class="footnote-reference brackets" href="#id10" id="id5" role="doc-noteref"><span class="fn-bracket">[</span>4<span class="fn-bracket">]</span></a></p>

  821. <p>Although powerful, implementing <a class="reference internal" href="#object.__reduce__" title="object.__reduce__"><code class="xref py py-meth docutils literal notranslate"><span class="pre">__reduce__()</span></code></a> directly in your classes is

  822. error prone.  For this reason, class designers should use the high-level

  823. interface (i.e., <a class="reference internal" href="#object.__getnewargs_ex__" title="object.__getnewargs_ex__"><code class="xref py py-meth docutils literal notranslate"><span class="pre">__getnewargs_ex__()</span></code></a>, <a class="reference internal" href="#object.__getstate__" title="object.__getstate__"><code class="xref py py-meth docutils literal notranslate"><span class="pre">__getstate__()</span></code></a> and

  824. <a class="reference internal" href="#object.__setstate__" title="object.__setstate__"><code class="xref py py-meth docutils literal notranslate"><span class="pre">__setstate__()</span></code></a>) whenever possible.  We will show, however, cases where

  825. using <code class="xref py py-meth docutils literal notranslate"><span class="pre">__reduce__()</span></code> is the only option or leads to more efficient pickling

  826. or both.</p>

  827. <dl class="py method">

  828. <dt class="sig sig-object py" id="object.__reduce__">

  829. <span class="sig-prename descclassname"><span class="pre">object.</span></span><span class="sig-name descname"><span class="pre">__reduce__</span></span><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="headerlink" href="#object.__reduce__" title="Link to this definition">¶</a></dt>

  830. <dd><p>The interface is currently defined as follows.  The <a class="reference internal" href="#object.__reduce__" title="object.__reduce__"><code class="xref py py-meth docutils literal notranslate"><span class="pre">__reduce__()</span></code></a> method

  831. takes no argument and shall return either a string or preferably a tuple (the

  832. returned object is often referred to as the “reduce value”).</p>

  833. <p>If a string is returned, the string should be interpreted as the name of a

  834. global variable.  It should be the object’s local name relative to its

  835. module; the pickle module searches the module namespace to determine the

  836. object’s module.  This behaviour is typically useful for singletons.</p>

  837. <p>When a tuple is returned, it must be between two and six items long.

  838. Optional items can either be omitted, or <code class="docutils literal notranslate"><span class="pre">None</span></code> can be provided as their

  839. value.  The semantics of each item are in order:</p>

  840. <ul>

  841. <li><p>A callable object that will be called to create the initial version of the

  842. object.</p></li>

  843. <li><p>A tuple of arguments for the callable object.  An empty tuple must be given

  844. if the callable does not accept any argument.</p></li>

  845. <li><p>Optionally, the object’s state, which will be passed to the object’s

  846. <a class="reference internal" href="#object.__setstate__" title="object.__setstate__"><code class="xref py py-meth docutils literal notranslate"><span class="pre">__setstate__()</span></code></a> method as previously described.  If the object has no

  847. such method then, the value must be a dictionary and it will be added to

  848. the object’s <a class="reference internal" href="stdtypes.html#object.__dict__" title="object.__dict__"><code class="xref py py-attr docutils literal notranslate"><span class="pre">__dict__</span></code></a> attribute.</p></li>

  849. <li><p>Optionally, an iterator (and not a sequence) yielding successive items.

  850. These items will be appended to the object either using

  851. <code class="docutils literal notranslate"><span class="pre">obj.append(item)</span></code> or, in batch, using <code class="docutils literal notranslate"><span class="pre">obj.extend(list_of_items)</span></code>.

  852. This is primarily used for list subclasses, but may be used by other

  853. classes as long as they have

  854. <a class="reference internal" href="stdtypes.html#typesseq-common"><span class="std std-ref">append and extend methods</span></a> with

  855. the appropriate signature.  (Whether <code class="xref py py-meth docutils literal notranslate"><span class="pre">append()</span></code> or <code class="xref py py-meth docutils literal notranslate"><span class="pre">extend()</span></code> is

  856. used depends on which pickle protocol version is used as well as the number

  857. of items to append, so both must be supported.)</p></li>

  858. <li><p>Optionally, an iterator (not a sequence) yielding successive key-value

  859. pairs.  These items will be stored to the object using <code class="docutils literal notranslate"><span class="pre">obj[key]</span> <span class="pre">=</span>

  860. <span class="pre">value</span></code>.  This is primarily used for dictionary subclasses, but may be used

  861. by other classes as long as they implement <a class="reference internal" href="../reference/datamodel.html#object.__setitem__" title="object.__setitem__"><code class="xref py py-meth docutils literal notranslate"><span class="pre">__setitem__()</span></code></a>.</p></li>

  862. <li><p>Optionally, a callable with a <code class="docutils literal notranslate"><span class="pre">(obj,</span> <span class="pre">state)</span></code> signature. This

  863. callable allows the user to programmatically control the state-updating

  864. behavior of a specific object, instead of using <code class="docutils literal notranslate"><span class="pre">obj</span></code>’s static

  865. <a class="reference internal" href="#object.__setstate__" title="object.__setstate__"><code class="xref py py-meth docutils literal notranslate"><span class="pre">__setstate__()</span></code></a> method. If not <code class="docutils literal notranslate"><span class="pre">None</span></code>, this callable will have

  866. priority over <code class="docutils literal notranslate"><span class="pre">obj</span></code>’s <a class="reference internal" href="#object.__setstate__" title="object.__setstate__"><code class="xref py py-meth docutils literal notranslate"><span class="pre">__setstate__()</span></code></a>.</p>

  867. <div class="versionadded">

  868. <p><span class="versionmodified added">New in version 3.8: </span>The optional sixth tuple item, <code class="docutils literal notranslate"><span class="pre">(obj,</span> <span class="pre">state)</span></code>, was added.</p>

  869. </div>

  870. </li>

  871. </ul>

  872. </dd></dl>

  873. 

  874. <dl class="py method">

  875. <dt class="sig sig-object py" id="object.__reduce_ex__">

  876. <span class="sig-prename descclassname"><span class="pre">object.</span></span><span class="sig-name descname"><span class="pre">__reduce_ex__</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">protocol</span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#object.__reduce_ex__" title="Link to this definition">¶</a></dt>

  877. <dd><p>Alternatively, a <a class="reference internal" href="#object.__reduce_ex__" title="object.__reduce_ex__"><code class="xref py py-meth docutils literal notranslate"><span class="pre">__reduce_ex__()</span></code></a> method may be defined.  The only

  878. difference is this method should take a single integer argument, the protocol

  879. version.  When defined, pickle will prefer it over the <a class="reference internal" href="#object.__reduce__" title="object.__reduce__"><code class="xref py py-meth docutils literal notranslate"><span class="pre">__reduce__()</span></code></a>

  880. method.  In addition, <a class="reference internal" href="#object.__reduce__" title="object.__reduce__"><code class="xref py py-meth docutils literal notranslate"><span class="pre">__reduce__()</span></code></a> automatically becomes a synonym for

  881. the extended version.  The main use for this method is to provide

  882. backwards-compatible reduce values for older Python releases.</p>

  883. </dd></dl>

  884. 

  885. <section id="persistence-of-external-objects">

  886. <span id="pickle-persistent"></span><h3>Persistence of External Objects<a class="headerlink" href="#persistence-of-external-objects" title="Link to this heading">¶</a></h3>

  887. <p id="index-6">For the benefit of object persistence, the <a class="reference internal" href="#module-pickle" title="pickle: Convert Python objects to streams of bytes and back."><code class="xref py py-mod docutils literal notranslate"><span class="pre">pickle</span></code></a> module supports the

  888. notion of a reference to an object outside the pickled data stream.  Such

  889. objects are referenced by a persistent ID, which should be either a string of

  890. alphanumeric characters (for protocol 0) <a class="footnote-reference brackets" href="#id11" id="id6" role="doc-noteref"><span class="fn-bracket">[</span>5<span class="fn-bracket">]</span></a> or just an arbitrary object (for

  891. any newer protocol).</p>

  892. <p>The resolution of such persistent IDs is not defined by the <a class="reference internal" href="#module-pickle" title="pickle: Convert Python objects to streams of bytes and back."><code class="xref py py-mod docutils literal notranslate"><span class="pre">pickle</span></code></a>

  893. module; it will delegate this resolution to the user-defined methods on the

  894. pickler and unpickler, <a class="reference internal" href="#pickle.Pickler.persistent_id" title="pickle.Pickler.persistent_id"><code class="xref py py-meth docutils literal notranslate"><span class="pre">persistent_id()</span></code></a> and

  895. <a class="reference internal" href="#pickle.Unpickler.persistent_load" title="pickle.Unpickler.persistent_load"><code class="xref py py-meth docutils literal notranslate"><span class="pre">persistent_load()</span></code></a> respectively.</p>

  896. <p>To pickle objects that have an external persistent ID, the pickler must have a

  897. custom <a class="reference internal" href="#pickle.Pickler.persistent_id" title="pickle.Pickler.persistent_id"><code class="xref py py-meth docutils literal notranslate"><span class="pre">persistent_id()</span></code></a> method that takes an object as an

  898. argument and returns either <code class="docutils literal notranslate"><span class="pre">None</span></code> or the persistent ID for that object.

  899. When <code class="docutils literal notranslate"><span class="pre">None</span></code> is returned, the pickler simply pickles the object as normal.

  900. When a persistent ID string is returned, the pickler will pickle that object,

  901. along with a marker so that the unpickler will recognize it as a persistent ID.</p>

  902. <p>To unpickle external objects, the unpickler must have a custom

  903. <a class="reference internal" href="#pickle.Unpickler.persistent_load" title="pickle.Unpickler.persistent_load"><code class="xref py py-meth docutils literal notranslate"><span class="pre">persistent_load()</span></code></a> method that takes a persistent ID object and

  904. returns the referenced object.</p>

  905. <p>Here is a comprehensive example presenting how persistent ID can be used to

  906. pickle external objects by reference.</p>

  907. <div class="highlight-python3 notranslate"><div class="highlight"><pre><span></span><span class="c1"># Simple example presenting how persistent ID can be used to pickle</span>

  908. <span class="c1"># external objects by reference.</span>

  909. 

  910. <span class="kn">import</span> <span class="nn">pickle</span>

  911. <span class="kn">import</span> <span class="nn">sqlite3</span>

  912. <span class="kn">from</span> <span class="nn">collections</span> <span class="kn">import</span> <span class="n">namedtuple</span>

  913. 

  914. <span class="c1"># Simple class representing a record in our database.</span>

  915. <span class="n">MemoRecord</span> <span class="o">=</span> <span class="n">namedtuple</span><span class="p">(</span><span class="s2">&quot;MemoRecord&quot;</span><span class="p">,</span> <span class="s2">&quot;key, task&quot;</span><span class="p">)</span>

  916. 

  917. <span class="k">class</span> <span class="nc">DBPickler</span><span class="p">(</span><span class="n">pickle</span><span class="o">.</span><span class="n">Pickler</span><span class="p">):</span>

  918. 

  919.     <span class="k">def</span> <span class="nf">persistent_id</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">obj</span><span class="p">):</span>

  920.         <span class="c1"># Instead of pickling MemoRecord as a regular class instance, we emit a</span>

  921.         <span class="c1"># persistent ID.</span>

  922.         <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">obj</span><span class="p">,</span> <span class="n">MemoRecord</span><span class="p">):</span>

  923.             <span class="c1"># Here, our persistent ID is simply a tuple, containing a tag and a</span>

  924.             <span class="c1"># key, which refers to a specific record in the database.</span>

  925.             <span class="k">return</span> <span class="p">(</span><span class="s2">&quot;MemoRecord&quot;</span><span class="p">,</span> <span class="n">obj</span><span class="o">.</span><span class="n">key</span><span class="p">)</span>

  926.         <span class="k">else</span><span class="p">:</span>

  927.             <span class="c1"># If obj does not have a persistent ID, return None. This means obj</span>

  928.             <span class="c1"># needs to be pickled as usual.</span>

  929.             <span class="k">return</span> <span class="kc">None</span>

  930. 

  931. 

  932. <span class="k">class</span> <span class="nc">DBUnpickler</span><span class="p">(</span><span class="n">pickle</span><span class="o">.</span><span class="n">Unpickler</span><span class="p">):</span>

  933. 

  934.     <span class="k">def</span> <span class="fm">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">file</span><span class="p">,</span> <span class="n">connection</span><span class="p">):</span>

  935.         <span class="nb">super</span><span class="p">()</span><span class="o">.</span><span class="fm">__init__</span><span class="p">(</span><span class="n">file</span><span class="p">)</span>

  936.         <span class="bp">self</span><span class="o">.</span><span class="n">connection</span> <span class="o">=</span> <span class="n">connection</span>

  937. 

  938.     <span class="k">def</span> <span class="nf">persistent_load</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">pid</span><span class="p">):</span>

  939.         <span class="c1"># This method is invoked whenever a persistent ID is encountered.</span>

  940.         <span class="c1"># Here, pid is the tuple returned by DBPickler.</span>

  941.         <span class="n">cursor</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">connection</span><span class="o">.</span><span class="n">cursor</span><span class="p">()</span>

  942.         <span class="n">type_tag</span><span class="p">,</span> <span class="n">key_id</span> <span class="o">=</span> <span class="n">pid</span>

  943.         <span class="k">if</span> <span class="n">type_tag</span> <span class="o">==</span> <span class="s2">&quot;MemoRecord&quot;</span><span class="p">:</span>

  944.             <span class="c1"># Fetch the referenced record from the database and return it.</span>

  945.             <span class="n">cursor</span><span class="o">.</span><span class="n">execute</span><span class="p">(</span><span class="s2">&quot;SELECT * FROM memos WHERE key=?&quot;</span><span class="p">,</span> <span class="p">(</span><span class="nb">str</span><span class="p">(</span><span class="n">key_id</span><span class="p">),))</span>

  946.             <span class="n">key</span><span class="p">,</span> <span class="n">task</span> <span class="o">=</span> <span class="n">cursor</span><span class="o">.</span><span class="n">fetchone</span><span class="p">()</span>

  947.             <span class="k">return</span> <span class="n">MemoRecord</span><span class="p">(</span><span class="n">key</span><span class="p">,</span> <span class="n">task</span><span class="p">)</span>

  948.         <span class="k">else</span><span class="p">:</span>

  949.             <span class="c1"># Always raises an error if you cannot return the correct object.</span>

  950.             <span class="c1"># Otherwise, the unpickler will think None is the object referenced</span>

  951.             <span class="c1"># by the persistent ID.</span>

  952.             <span class="k">raise</span> <span class="n">pickle</span><span class="o">.</span><span class="n">UnpicklingError</span><span class="p">(</span><span class="s2">&quot;unsupported persistent object&quot;</span><span class="p">)</span>

  953. 

  954. 

  955. <span class="k">def</span> <span class="nf">main</span><span class="p">():</span>

  956.     <span class="kn">import</span> <span class="nn">io</span>

  957.     <span class="kn">import</span> <span class="nn">pprint</span>

  958. 

  959.     <span class="c1"># Initialize and populate our database.</span>

  960.     <span class="n">conn</span> <span class="o">=</span> <span class="n">sqlite3</span><span class="o">.</span><span class="n">connect</span><span class="p">(</span><span class="s2">&quot;:memory:&quot;</span><span class="p">)</span>

  961.     <span class="n">cursor</span> <span class="o">=</span> <span class="n">conn</span><span class="o">.</span><span class="n">cursor</span><span class="p">()</span>

  962.     <span class="n">cursor</span><span class="o">.</span><span class="n">execute</span><span class="p">(</span><span class="s2">&quot;CREATE TABLE memos(key INTEGER PRIMARY KEY, task TEXT)&quot;</span><span class="p">)</span>

  963.     <span class="n">tasks</span> <span class="o">=</span> <span class="p">(</span>

  964.         <span class="s1">&#39;give food to fish&#39;</span><span class="p">,</span>

  965.         <span class="s1">&#39;prepare group meeting&#39;</span><span class="p">,</span>

  966.         <span class="s1">&#39;fight with a zebra&#39;</span><span class="p">,</span>

  967.         <span class="p">)</span>

  968.     <span class="k">for</span> <span class="n">task</span> <span class="ow">in</span> <span class="n">tasks</span><span class="p">:</span>

  969.         <span class="n">cursor</span><span class="o">.</span><span class="n">execute</span><span class="p">(</span><span class="s2">&quot;INSERT INTO memos VALUES(NULL, ?)&quot;</span><span class="p">,</span> <span class="p">(</span><span class="n">task</span><span class="p">,))</span>

  970. 

  971.     <span class="c1"># Fetch the records to be pickled.</span>

  972.     <span class="n">cursor</span><span class="o">.</span><span class="n">execute</span><span class="p">(</span><span class="s2">&quot;SELECT * FROM memos&quot;</span><span class="p">)</span>

  973.     <span class="n">memos</span> <span class="o">=</span> <span class="p">[</span><span class="n">MemoRecord</span><span class="p">(</span><span class="n">key</span><span class="p">,</span> <span class="n">task</span><span class="p">)</span> <span class="k">for</span> <span class="n">key</span><span class="p">,</span> <span class="n">task</span> <span class="ow">in</span> <span class="n">cursor</span><span class="p">]</span>

  974.     <span class="c1"># Save the records using our custom DBPickler.</span>

  975.     <span class="n">file</span> <span class="o">=</span> <span class="n">io</span><span class="o">.</span><span class="n">BytesIO</span><span class="p">()</span>

  976.     <span class="n">DBPickler</span><span class="p">(</span><span class="n">file</span><span class="p">)</span><span class="o">.</span><span class="n">dump</span><span class="p">(</span><span class="n">memos</span><span class="p">)</span>

  977. 

  978.     <span class="nb">print</span><span class="p">(</span><span class="s2">&quot;Pickled records:&quot;</span><span class="p">)</span>

  979.     <span class="n">pprint</span><span class="o">.</span><span class="n">pprint</span><span class="p">(</span><span class="n">memos</span><span class="p">)</span>

  980. 

  981.     <span class="c1"># Update a record, just for good measure.</span>

  982.     <span class="n">cursor</span><span class="o">.</span><span class="n">execute</span><span class="p">(</span><span class="s2">&quot;UPDATE memos SET task=&#39;learn italian&#39; WHERE key=1&quot;</span><span class="p">)</span>

  983. 

  984.     <span class="c1"># Load the records from the pickle data stream.</span>

  985.     <span class="n">file</span><span class="o">.</span><span class="n">seek</span><span class="p">(</span><span class="mi">0</span><span class="p">)</span>

  986.     <span class="n">memos</span> <span class="o">=</span> <span class="n">DBUnpickler</span><span class="p">(</span><span class="n">file</span><span class="p">,</span> <span class="n">conn</span><span class="p">)</span><span class="o">.</span><span class="n">load</span><span class="p">()</span>

  987. 

  988.     <span class="nb">print</span><span class="p">(</span><span class="s2">&quot;Unpickled records:&quot;</span><span class="p">)</span>

  989.     <span class="n">pprint</span><span class="o">.</span><span class="n">pprint</span><span class="p">(</span><span class="n">memos</span><span class="p">)</span>

  990. 

  991. 

  992. <span class="k">if</span> <span class="vm">__name__</span> <span class="o">==</span> <span class="s1">&#39;__main__&#39;</span><span class="p">:</span>

  993.     <span class="n">main</span><span class="p">()</span>

  994. </pre></div>

  995. </div>

  996. </section>

  997. <section id="dispatch-tables">

  998. <span id="pickle-dispatch"></span><h3>Dispatch Tables<a class="headerlink" href="#dispatch-tables" title="Link to this heading">¶</a></h3>

  999. <p>If one wants to customize pickling of some classes without disturbing

  1000. any other code which depends on pickling, then one can create a

  1001. pickler with a private dispatch table.</p>

  1002. <p>The global dispatch table managed by the <a class="reference internal" href="copyreg.html#module-copyreg" title="copyreg: Register pickle support functions."><code class="xref py py-mod docutils literal notranslate"><span class="pre">copyreg</span></code></a> module is

  1003. available as <code class="xref py py-data docutils literal notranslate"><span class="pre">copyreg.dispatch_table</span></code>.  Therefore, one may

  1004. choose to use a modified copy of <code class="xref py py-data docutils literal notranslate"><span class="pre">copyreg.dispatch_table</span></code> as a

  1005. private dispatch table.</p>

  1006. <p>For example</p>

  1007. <div class="highlight-python3 notranslate"><div class="highlight"><pre><span></span><span class="n">f</span> <span class="o">=</span> <span class="n">io</span><span class="o">.</span><span class="n">BytesIO</span><span class="p">()</span>

  1008. <span class="n">p</span> <span class="o">=</span> <span class="n">pickle</span><span class="o">.</span><span class="n">Pickler</span><span class="p">(</span><span class="n">f</span><span class="p">)</span>

  1009. <span class="n">p</span><span class="o">.</span><span class="n">dispatch_table</span> <span class="o">=</span> <span class="n">copyreg</span><span class="o">.</span><span class="n">dispatch_table</span><span class="o">.</span><span class="n">copy</span><span class="p">()</span>

  1010. <span class="n">p</span><span class="o">.</span><span class="n">dispatch_table</span><span class="p">[</span><span class="n">SomeClass</span><span class="p">]</span> <span class="o">=</span> <span class="n">reduce_SomeClass</span>

  1011. </pre></div>

  1012. </div>

  1013. <p>creates an instance of <a class="reference internal" href="#pickle.Pickler" title="pickle.Pickler"><code class="xref py py-class docutils literal notranslate"><span class="pre">pickle.Pickler</span></code></a> with a private dispatch

  1014. table which handles the <code class="docutils literal notranslate"><span class="pre">SomeClass</span></code> class specially.  Alternatively,

  1015. the code</p>

  1016. <div class="highlight-python3 notranslate"><div class="highlight"><pre><span></span><span class="k">class</span> <span class="nc">MyPickler</span><span class="p">(</span><span class="n">pickle</span><span class="o">.</span><span class="n">Pickler</span><span class="p">):</span>

  1017.     <span class="n">dispatch_table</span> <span class="o">=</span> <span class="n">copyreg</span><span class="o">.</span><span class="n">dispatch_table</span><span class="o">.</span><span class="n">copy</span><span class="p">()</span>

  1018.     <span class="n">dispatch_table</span><span class="p">[</span><span class="n">SomeClass</span><span class="p">]</span> <span class="o">=</span> <span class="n">reduce_SomeClass</span>

  1019. <span class="n">f</span> <span class="o">=</span> <span class="n">io</span><span class="o">.</span><span class="n">BytesIO</span><span class="p">()</span>

  1020. <span class="n">p</span> <span class="o">=</span> <span class="n">MyPickler</span><span class="p">(</span><span class="n">f</span><span class="p">)</span>

  1021. </pre></div>

  1022. </div>

  1023. <p>does the same but all instances of <code class="docutils literal notranslate"><span class="pre">MyPickler</span></code> will by default

  1024. share the private dispatch table.  On the other hand, the code</p>

  1025. <div class="highlight-python3 notranslate"><div class="highlight"><pre><span></span><span class="n">copyreg</span><span class="o">.</span><span class="n">pickle</span><span class="p">(</span><span class="n">SomeClass</span><span class="p">,</span> <span class="n">reduce_SomeClass</span><span class="p">)</span>

  1026. <span class="n">f</span> <span class="o">=</span> <span class="n">io</span><span class="o">.</span><span class="n">BytesIO</span><span class="p">()</span>

  1027. <span class="n">p</span> <span class="o">=</span> <span class="n">pickle</span><span class="o">.</span><span class="n">Pickler</span><span class="p">(</span><span class="n">f</span><span class="p">)</span>

  1028. </pre></div>

  1029. </div>

  1030. <p>modifies the global dispatch table shared by all users of the <a class="reference internal" href="copyreg.html#module-copyreg" title="copyreg: Register pickle support functions."><code class="xref py py-mod docutils literal notranslate"><span class="pre">copyreg</span></code></a> module.</p>

  1031. </section>

  1032. <section id="handling-stateful-objects">

  1033. <span id="pickle-state"></span><h3>Handling Stateful Objects<a class="headerlink" href="#handling-stateful-objects" title="Link to this heading">¶</a></h3>

  1034. <p id="index-7">Here’s an example that shows how to modify pickling behavior for a class.

  1035. The <code class="xref py py-class docutils literal notranslate"><span class="pre">TextReader</span></code> class below opens a text file, and returns the line number and

  1036. line contents each time its <code class="xref py py-meth docutils literal notranslate"><span class="pre">readline()</span></code> method is called. If a

  1037. <code class="xref py py-class docutils literal notranslate"><span class="pre">TextReader</span></code> instance is pickled, all attributes <em>except</em> the file object

  1038. member are saved. When the instance is unpickled, the file is reopened, and

  1039. reading resumes from the last location. The <code class="xref py py-meth docutils literal notranslate"><span class="pre">__setstate__()</span></code> and

  1040. <code class="xref py py-meth docutils literal notranslate"><span class="pre">__getstate__()</span></code> methods are used to implement this behavior.</p>

  1041. <div class="highlight-python3 notranslate"><div class="highlight"><pre><span></span><span class="k">class</span> <span class="nc">TextReader</span><span class="p">:</span>

  1042. <span class="w">    </span><span class="sd">&quot;&quot;&quot;Print and number lines in a text file.&quot;&quot;&quot;</span>

  1043. 

  1044.     <span class="k">def</span> <span class="fm">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">filename</span><span class="p">):</span>

  1045.         <span class="bp">self</span><span class="o">.</span><span class="n">filename</span> <span class="o">=</span> <span class="n">filename</span>

  1046.         <span class="bp">self</span><span class="o">.</span><span class="n">file</span> <span class="o">=</span> <span class="nb">open</span><span class="p">(</span><span class="n">filename</span><span class="p">)</span>

  1047.         <span class="bp">self</span><span class="o">.</span><span class="n">lineno</span> <span class="o">=</span> <span class="mi">0</span>

  1048. 

  1049.     <span class="k">def</span> <span class="nf">readline</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>

  1050.         <span class="bp">self</span><span class="o">.</span><span class="n">lineno</span> <span class="o">+=</span> <span class="mi">1</span>

  1051.         <span class="n">line</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">file</span><span class="o">.</span><span class="n">readline</span><span class="p">()</span>

  1052.         <span class="k">if</span> <span class="ow">not</span> <span class="n">line</span><span class="p">:</span>

  1053.             <span class="k">return</span> <span class="kc">None</span>

  1054.         <span class="k">if</span> <span class="n">line</span><span class="o">.</span><span class="n">endswith</span><span class="p">(</span><span class="s1">&#39;</span><span class="se">\n</span><span class="s1">&#39;</span><span class="p">):</span>

  1055.             <span class="n">line</span> <span class="o">=</span> <span class="n">line</span><span class="p">[:</span><span class="o">-</span><span class="mi">1</span><span class="p">]</span>

  1056.         <span class="k">return</span> <span class="s2">&quot;</span><span class="si">%i</span><span class="s2">: </span><span class="si">%s</span><span class="s2">&quot;</span> <span class="o">%</span> <span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">lineno</span><span class="p">,</span> <span class="n">line</span><span class="p">)</span>

  1057. 

  1058.     <span class="k">def</span> <span class="nf">__getstate__</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>

  1059.         <span class="c1"># Copy the object&#39;s state from self.__dict__ which contains</span>

  1060.         <span class="c1"># all our instance attributes. Always use the dict.copy()</span>

  1061.         <span class="c1"># method to avoid modifying the original state.</span>

  1062.         <span class="n">state</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="vm">__dict__</span><span class="o">.</span><span class="n">copy</span><span class="p">()</span>

  1063.         <span class="c1"># Remove the unpicklable entries.</span>

  1064.         <span class="k">del</span> <span class="n">state</span><span class="p">[</span><span class="s1">&#39;file&#39;</span><span class="p">]</span>

  1065.         <span class="k">return</span> <span class="n">state</span>

  1066. 

  1067.     <span class="k">def</span> <span class="nf">__setstate__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">state</span><span class="p">):</span>

  1068.         <span class="c1"># Restore instance attributes (i.e., filename and lineno).</span>

  1069.         <span class="bp">self</span><span class="o">.</span><span class="vm">__dict__</span><span class="o">.</span><span class="n">update</span><span class="p">(</span><span class="n">state</span><span class="p">)</span>

  1070.         <span class="c1"># Restore the previously opened file&#39;s state. To do so, we need to</span>

  1071.         <span class="c1"># reopen it and read from it until the line count is restored.</span>

  1072.         <span class="n">file</span> <span class="o">=</span> <span class="nb">open</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">filename</span><span class="p">)</span>

  1073.         <span class="k">for</span> <span class="n">_</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">lineno</span><span class="p">):</span>

  1074.             <span class="n">file</span><span class="o">.</span><span class="n">readline</span><span class="p">()</span>

  1075.         <span class="c1"># Finally, save the file.</span>

  1076.         <span class="bp">self</span><span class="o">.</span><span class="n">file</span> <span class="o">=</span> <span class="n">file</span>

  1077. </pre></div>

  1078. </div>

  1079. <p>A sample usage might be something like this:</p>

  1080. <div class="highlight-python3 notranslate"><div class="highlight"><pre><span></span><span class="gp">&gt;&gt;&gt; </span><span class="n">reader</span> <span class="o">=</span> <span class="n">TextReader</span><span class="p">(</span><span class="s2">&quot;hello.txt&quot;</span><span class="p">)</span>

  1081. <span class="gp">&gt;&gt;&gt; </span><span class="n">reader</span><span class="o">.</span><span class="n">readline</span><span class="p">()</span>

  1082. <span class="go">&#39;1: Hello world!&#39;</span>

  1083. <span class="gp">&gt;&gt;&gt; </span><span class="n">reader</span><span class="o">.</span><span class="n">readline</span><span class="p">()</span>

  1084. <span class="go">&#39;2: I am line number two.&#39;</span>

  1085. <span class="gp">&gt;&gt;&gt; </span><span class="n">new_reader</span> <span class="o">=</span> <span class="n">pickle</span><span class="o">.</span><span class="n">loads</span><span class="p">(</span><span class="n">pickle</span><span class="o">.</span><span class="n">dumps</span><span class="p">(</span><span class="n">reader</span><span class="p">))</span>

  1086. <span class="gp">&gt;&gt;&gt; </span><span class="n">new_reader</span><span class="o">.</span><span class="n">readline</span><span class="p">()</span>

  1087. <span class="go">&#39;3: Goodbye!&#39;</span>

  1088. </pre></div>

  1089. </div>

  1090. </section>

  1091. </section>

  1092. <section id="custom-reduction-for-types-functions-and-other-objects">

  1093. <span id="reducer-override"></span><h2>Custom Reduction for Types, Functions, and Other Objects<a class="headerlink" href="#custom-reduction-for-types-functions-and-other-objects" title="Link to this heading">¶</a></h2>

  1094. <div class="versionadded">

  1095. <p><span class="versionmodified added">New in version 3.8.</span></p>

  1096. </div>

  1097. <p>Sometimes, <a class="reference internal" href="#pickle.Pickler.dispatch_table" title="pickle.Pickler.dispatch_table"><code class="xref py py-attr docutils literal notranslate"><span class="pre">dispatch_table</span></code></a> may not be flexible enough.

  1098. In particular we may want to customize pickling based on another criterion

  1099. than the object’s type, or we may want to customize the pickling of

  1100. functions and classes.</p>

  1101. <p>For those cases, it is possible to subclass from the <a class="reference internal" href="#pickle.Pickler" title="pickle.Pickler"><code class="xref py py-class docutils literal notranslate"><span class="pre">Pickler</span></code></a> class and

  1102. implement a <a class="reference internal" href="#pickle.Pickler.reducer_override" title="pickle.Pickler.reducer_override"><code class="xref py py-meth docutils literal notranslate"><span class="pre">reducer_override()</span></code></a> method. This method can return an

  1103. arbitrary reduction tuple (see <a class="reference internal" href="#object.__reduce__" title="object.__reduce__"><code class="xref py py-meth docutils literal notranslate"><span class="pre">__reduce__()</span></code></a>). It can alternatively return

  1104. <a class="reference internal" href="constants.html#NotImplemented" title="NotImplemented"><code class="xref py py-data docutils literal notranslate"><span class="pre">NotImplemented</span></code></a> to fallback to the traditional behavior.</p>

  1105. <p>If both the <a class="reference internal" href="#pickle.Pickler.dispatch_table" title="pickle.Pickler.dispatch_table"><code class="xref py py-attr docutils literal notranslate"><span class="pre">dispatch_table</span></code></a> and

  1106. <a class="reference internal" href="#pickle.Pickler.reducer_override" title="pickle.Pickler.reducer_override"><code class="xref py py-meth docutils literal notranslate"><span class="pre">reducer_override()</span></code></a> are defined, then

  1107. <a class="reference internal" href="#pickle.Pickler.reducer_override" title="pickle.Pickler.reducer_override"><code class="xref py py-meth docutils literal notranslate"><span class="pre">reducer_override()</span></code></a> method takes priority.</p>

  1108. <div class="admonition note">

  1109. <p class="admonition-title">Note</p>

  1110. <p>For performance reasons, <a class="reference internal" href="#pickle.Pickler.reducer_override" title="pickle.Pickler.reducer_override"><code class="xref py py-meth docutils literal notranslate"><span class="pre">reducer_override()</span></code></a> may not be

  1111. called for the following objects: <code class="docutils literal notranslate"><span class="pre">None</span></code>, <code class="docutils literal notranslate"><span class="pre">True</span></code>, <code class="docutils literal notranslate"><span class="pre">False</span></code>, and

  1112. exact instances of <a class="reference internal" href="functions.html#int" title="int"><code class="xref py py-class docutils literal notranslate"><span class="pre">int</span></code></a>, <a class="reference internal" href="functions.html#float" title="float"><code class="xref py py-class docutils literal notranslate"><span class="pre">float</span></code></a>, <a class="reference internal" href="stdtypes.html#bytes" title="bytes"><code class="xref py py-class docutils literal notranslate"><span class="pre">bytes</span></code></a>,

  1113. <a class="reference internal" href="stdtypes.html#str" title="str"><code class="xref py py-class docutils literal notranslate"><span class="pre">str</span></code></a>, <a class="reference internal" href="stdtypes.html#dict" title="dict"><code class="xref py py-class docutils literal notranslate"><span class="pre">dict</span></code></a>, <a class="reference internal" href="stdtypes.html#set" title="set"><code class="xref py py-class docutils literal notranslate"><span class="pre">set</span></code></a>, <a class="reference internal" href="stdtypes.html#frozenset" title="frozenset"><code class="xref py py-class docutils literal notranslate"><span class="pre">frozenset</span></code></a>, <a class="reference internal" href="stdtypes.html#list" title="list"><code class="xref py py-class docutils literal notranslate"><span class="pre">list</span></code></a>

  1114. and <a class="reference internal" href="stdtypes.html#tuple" title="tuple"><code class="xref py py-class docutils literal notranslate"><span class="pre">tuple</span></code></a>.</p>

  1115. </div>

  1116. <p>Here is a simple example where we allow pickling and reconstructing

  1117. a given class:</p>

  1118. <div class="highlight-python3 notranslate"><div class="highlight"><pre><span></span><span class="kn">import</span> <span class="nn">io</span>

  1119. <span class="kn">import</span> <span class="nn">pickle</span>

  1120. 

  1121. <span class="k">class</span> <span class="nc">MyClass</span><span class="p">:</span>

  1122.     <span class="n">my_attribute</span> <span class="o">=</span> <span class="mi">1</span>

  1123. 

  1124. <span class="k">class</span> <span class="nc">MyPickler</span><span class="p">(</span><span class="n">pickle</span><span class="o">.</span><span class="n">Pickler</span><span class="p">):</span>

  1125.     <span class="k">def</span> <span class="nf">reducer_override</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">obj</span><span class="p">):</span>

  1126. <span class="w">        </span><span class="sd">&quot;&quot;&quot;Custom reducer for MyClass.&quot;&quot;&quot;</span>

  1127.         <span class="k">if</span> <span class="nb">getattr</span><span class="p">(</span><span class="n">obj</span><span class="p">,</span> <span class="s2">&quot;__name__&quot;</span><span class="p">,</span> <span class="kc">None</span><span class="p">)</span> <span class="o">==</span> <span class="s2">&quot;MyClass&quot;</span><span class="p">:</span>

  1128.             <span class="k">return</span> <span class="nb">type</span><span class="p">,</span> <span class="p">(</span><span class="n">obj</span><span class="o">.</span><span class="vm">__name__</span><span class="p">,</span> <span class="n">obj</span><span class="o">.</span><span class="vm">__bases__</span><span class="p">,</span>

  1129.                           <span class="p">{</span><span class="s1">&#39;my_attribute&#39;</span><span class="p">:</span> <span class="n">obj</span><span class="o">.</span><span class="n">my_attribute</span><span class="p">})</span>

  1130.         <span class="k">else</span><span class="p">:</span>

  1131.             <span class="c1"># For any other object, fallback to usual reduction</span>

  1132.             <span class="k">return</span> <span class="bp">NotImplemented</span>

  1133. 

  1134. <span class="n">f</span> <span class="o">=</span> <span class="n">io</span><span class="o">.</span><span class="n">BytesIO</span><span class="p">()</span>

  1135. <span class="n">p</span> <span class="o">=</span> <span class="n">MyPickler</span><span class="p">(</span><span class="n">f</span><span class="p">)</span>

  1136. <span class="n">p</span><span class="o">.</span><span class="n">dump</span><span class="p">(</span><span class="n">MyClass</span><span class="p">)</span>

  1137. 

  1138. <span class="k">del</span> <span class="n">MyClass</span>

  1139. 

  1140. <span class="n">unpickled_class</span> <span class="o">=</span> <span class="n">pickle</span><span class="o">.</span><span class="n">loads</span><span class="p">(</span><span class="n">f</span><span class="o">.</span><span class="n">getvalue</span><span class="p">())</span>

  1141. 

  1142. <span class="k">assert</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">unpickled_class</span><span class="p">,</span> <span class="nb">type</span><span class="p">)</span>

  1143. <span class="k">assert</span> <span class="n">unpickled_class</span><span class="o">.</span><span class="vm">__name__</span> <span class="o">==</span> <span class="s2">&quot;MyClass&quot;</span>

  1144. <span class="k">assert</span> <span class="n">unpickled_class</span><span class="o">.</span><span class="n">my_attribute</span> <span class="o">==</span> <span class="mi">1</span>

  1145. </pre></div>

  1146. </div>

  1147. </section>

  1148. <section id="out-of-band-buffers">

  1149. <span id="pickle-oob"></span><h2>Out-of-band Buffers<a class="headerlink" href="#out-of-band-buffers" title="Link to this heading">¶</a></h2>

  1150. <div class="versionadded">

  1151. <p><span class="versionmodified added">New in version 3.8.</span></p>

  1152. </div>

  1153. <p>In some contexts, the <a class="reference internal" href="#module-pickle" title="pickle: Convert Python objects to streams of bytes and back."><code class="xref py py-mod docutils literal notranslate"><span class="pre">pickle</span></code></a> module is used to transfer massive amounts

  1154. of data.  Therefore, it can be important to minimize the number of memory

  1155. copies, to preserve performance and resource consumption.  However, normal

  1156. operation of the <a class="reference internal" href="#module-pickle" title="pickle: Convert Python objects to streams of bytes and back."><code class="xref py py-mod docutils literal notranslate"><span class="pre">pickle</span></code></a> module, as it transforms a graph-like structure

  1157. of objects into a sequential stream of bytes, intrinsically involves copying

  1158. data to and from the pickle stream.</p>

  1159. <p>This constraint can be eschewed if both the <em>provider</em> (the implementation

  1160. of the object types to be transferred) and the <em>consumer</em> (the implementation

  1161. of the communications system) support the out-of-band transfer facilities

  1162. provided by pickle protocol 5 and higher.</p>

  1163. <section id="provider-api">

  1164. <h3>Provider API<a class="headerlink" href="#provider-api" title="Link to this heading">¶</a></h3>

  1165. <p>The large data objects to be pickled must implement a <a class="reference internal" href="#object.__reduce_ex__" title="object.__reduce_ex__"><code class="xref py py-meth docutils literal notranslate"><span class="pre">__reduce_ex__()</span></code></a>

  1166. method specialized for protocol 5 and higher, which returns a

  1167. <a class="reference internal" href="#pickle.PickleBuffer" title="pickle.PickleBuffer"><code class="xref py py-class docutils literal notranslate"><span class="pre">PickleBuffer</span></code></a> instance (instead of e.g. a <a class="reference internal" href="stdtypes.html#bytes" title="bytes"><code class="xref py py-class docutils literal notranslate"><span class="pre">bytes</span></code></a> object)

  1168. for any large data.</p>

  1169. <p>A <a class="reference internal" href="#pickle.PickleBuffer" title="pickle.PickleBuffer"><code class="xref py py-class docutils literal notranslate"><span class="pre">PickleBuffer</span></code></a> object <em>signals</em> that the underlying buffer is

  1170. eligible for out-of-band data transfer.  Those objects remain compatible

  1171. with normal usage of the <a class="reference internal" href="#module-pickle" title="pickle: Convert Python objects to streams of bytes and back."><code class="xref py py-mod docutils literal notranslate"><span class="pre">pickle</span></code></a> module.  However, consumers can also

  1172. opt-in to tell <a class="reference internal" href="#module-pickle" title="pickle: Convert Python objects to streams of bytes and back."><code class="xref py py-mod docutils literal notranslate"><span class="pre">pickle</span></code></a> that they will handle those buffers by

  1173. themselves.</p>

  1174. </section>

  1175. <section id="consumer-api">

  1176. <h3>Consumer API<a class="headerlink" href="#consumer-api" title="Link to this heading">¶</a></h3>

  1177. <p>A communications system can enable custom handling of the <a class="reference internal" href="#pickle.PickleBuffer" title="pickle.PickleBuffer"><code class="xref py py-class docutils literal notranslate"><span class="pre">PickleBuffer</span></code></a>

  1178. objects generated when serializing an object graph.</p>

  1179. <p>On the sending side, it needs to pass a <em>buffer_callback</em> argument to

  1180. <a class="reference internal" href="#pickle.Pickler" title="pickle.Pickler"><code class="xref py py-class docutils literal notranslate"><span class="pre">Pickler</span></code></a> (or to the <a class="reference internal" href="#pickle.dump" title="pickle.dump"><code class="xref py py-func docutils literal notranslate"><span class="pre">dump()</span></code></a> or <a class="reference internal" href="#pickle.dumps" title="pickle.dumps"><code class="xref py py-func docutils literal notranslate"><span class="pre">dumps()</span></code></a> function), which

  1181. will be called with each <a class="reference internal" href="#pickle.PickleBuffer" title="pickle.PickleBuffer"><code class="xref py py-class docutils literal notranslate"><span class="pre">PickleBuffer</span></code></a> generated while pickling

  1182. the object graph.  Buffers accumulated by the <em>buffer_callback</em> will not

  1183. see their data copied into the pickle stream, only a cheap marker will be

  1184. inserted.</p>

  1185. <p>On the receiving side, it needs to pass a <em>buffers</em> argument to

  1186. <a class="reference internal" href="#pickle.Unpickler" title="pickle.Unpickler"><code class="xref py py-class docutils literal notranslate"><span class="pre">Unpickler</span></code></a> (or to the <a class="reference internal" href="#pickle.load" title="pickle.load"><code class="xref py py-func docutils literal notranslate"><span class="pre">load()</span></code></a> or <a class="reference internal" href="#pickle.loads" title="pickle.loads"><code class="xref py py-func docutils literal notranslate"><span class="pre">loads()</span></code></a> function),

  1187. which is an iterable of the buffers which were passed to <em>buffer_callback</em>.

  1188. That iterable should produce buffers in the same order as they were passed

  1189. to <em>buffer_callback</em>.  Those buffers will provide the data expected by the

  1190. reconstructors of the objects whose pickling produced the original

  1191. <a class="reference internal" href="#pickle.PickleBuffer" title="pickle.PickleBuffer"><code class="xref py py-class docutils literal notranslate"><span class="pre">PickleBuffer</span></code></a> objects.</p>

  1192. <p>Between the sending side and the receiving side, the communications system

  1193. is free to implement its own transfer mechanism for out-of-band buffers.

  1194. Potential optimizations include the use of shared memory or datatype-dependent

  1195. compression.</p>

  1196. </section>

  1197. <section id="example">

  1198. <h3>Example<a class="headerlink" href="#example" title="Link to this heading">¶</a></h3>

  1199. <p>Here is a trivial example where we implement a <a class="reference internal" href="stdtypes.html#bytearray" title="bytearray"><code class="xref py py-class docutils literal notranslate"><span class="pre">bytearray</span></code></a> subclass

  1200. able to participate in out-of-band buffer pickling:</p>

  1201. <div class="highlight-python3 notranslate"><div class="highlight"><pre><span></span><span class="k">class</span> <span class="nc">ZeroCopyByteArray</span><span class="p">(</span><span class="nb">bytearray</span><span class="p">):</span>

  1202. 

  1203.     <span class="k">def</span> <span class="nf">__reduce_ex__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">protocol</span><span class="p">):</span>

  1204.         <span class="k">if</span> <span class="n">protocol</span> <span class="o">&gt;=</span> <span class="mi">5</span><span class="p">:</span>

  1205.             <span class="k">return</span> <span class="nb">type</span><span class="p">(</span><span class="bp">self</span><span class="p">)</span><span class="o">.</span><span class="n">_reconstruct</span><span class="p">,</span> <span class="p">(</span><span class="n">PickleBuffer</span><span class="p">(</span><span class="bp">self</span><span class="p">),),</span> <span class="kc">None</span>

  1206.         <span class="k">else</span><span class="p">:</span>

  1207.             <span class="c1"># PickleBuffer is forbidden with pickle protocols &lt;= 4.</span>

  1208.             <span class="k">return</span> <span class="nb">type</span><span class="p">(</span><span class="bp">self</span><span class="p">)</span><span class="o">.</span><span class="n">_reconstruct</span><span class="p">,</span> <span class="p">(</span><span class="nb">bytearray</span><span class="p">(</span><span class="bp">self</span><span class="p">),)</span>

  1209. 

  1210.     <span class="nd">@classmethod</span>

  1211.     <span class="k">def</span> <span class="nf">_reconstruct</span><span class="p">(</span><span class="bp">cls</span><span class="p">,</span> <span class="n">obj</span><span class="p">):</span>

  1212.         <span class="k">with</span> <span class="nb">memoryview</span><span class="p">(</span><span class="n">obj</span><span class="p">)</span> <span class="k">as</span> <span class="n">m</span><span class="p">:</span>

  1213.             <span class="c1"># Get a handle over the original buffer object</span>

  1214.             <span class="n">obj</span> <span class="o">=</span> <span class="n">m</span><span class="o">.</span><span class="n">obj</span>

  1215.             <span class="k">if</span> <span class="nb">type</span><span class="p">(</span><span class="n">obj</span><span class="p">)</span> <span class="ow">is</span> <span class="bp">cls</span><span class="p">:</span>

  1216.                 <span class="c1"># Original buffer object is a ZeroCopyByteArray, return it</span>

  1217.                 <span class="c1"># as-is.</span>

  1218.                 <span class="k">return</span> <span class="n">obj</span>

  1219.             <span class="k">else</span><span class="p">:</span>

  1220.                 <span class="k">return</span> <span class="bp">cls</span><span class="p">(</span><span class="n">obj</span><span class="p">)</span>

  1221. </pre></div>

  1222. </div>

  1223. <p>The reconstructor (the <code class="docutils literal notranslate"><span class="pre">_reconstruct</span></code> class method) returns the buffer’s

  1224. providing object if it has the right type.  This is an easy way to simulate

  1225. zero-copy behaviour on this toy example.</p>

  1226. <p>On the consumer side, we can pickle those objects the usual way, which

  1227. when unserialized will give us a copy of the original object:</p>

  1228. <div class="highlight-python3 notranslate"><div class="highlight"><pre><span></span><span class="n">b</span> <span class="o">=</span> <span class="n">ZeroCopyByteArray</span><span class="p">(</span><span class="sa">b</span><span class="s2">&quot;abc&quot;</span><span class="p">)</span>

  1229. <span class="n">data</span> <span class="o">=</span> <span class="n">pickle</span><span class="o">.</span><span class="n">dumps</span><span class="p">(</span><span class="n">b</span><span class="p">,</span> <span class="n">protocol</span><span class="o">=</span><span class="mi">5</span><span class="p">)</span>

  1230. <span class="n">new_b</span> <span class="o">=</span> <span class="n">pickle</span><span class="o">.</span><span class="n">loads</span><span class="p">(</span><span class="n">data</span><span class="p">)</span>

  1231. <span class="nb">print</span><span class="p">(</span><span class="n">b</span> <span class="o">==</span> <span class="n">new_b</span><span class="p">)</span>  <span class="c1"># True</span>

  1232. <span class="nb">print</span><span class="p">(</span><span class="n">b</span> <span class="ow">is</span> <span class="n">new_b</span><span class="p">)</span>  <span class="c1"># False: a copy was made</span>

  1233. </pre></div>

  1234. </div>

  1235. <p>But if we pass a <em>buffer_callback</em> and then give back the accumulated

  1236. buffers when unserializing, we are able to get back the original object:</p>

  1237. <div class="highlight-python3 notranslate"><div class="highlight"><pre><span></span><span class="n">b</span> <span class="o">=</span> <span class="n">ZeroCopyByteArray</span><span class="p">(</span><span class="sa">b</span><span class="s2">&quot;abc&quot;</span><span class="p">)</span>

  1238. <span class="n">buffers</span> <span class="o">=</span> <span class="p">[]</span>

  1239. <span class="n">data</span> <span class="o">=</span> <span class="n">pickle</span><span class="o">.</span><span class="n">dumps</span><span class="p">(</span><span class="n">b</span><span class="p">,</span> <span class="n">protocol</span><span class="o">=</span><span class="mi">5</span><span class="p">,</span> <span class="n">buffer_callback</span><span class="o">=</span><span class="n">buffers</span><span class="o">.</span><span class="n">append</span><span class="p">)</span>

  1240. <span class="n">new_b</span> <span class="o">=</span> <span class="n">pickle</span><span class="o">.</span><span class="n">loads</span><span class="p">(</span><span class="n">data</span><span class="p">,</span> <span class="n">buffers</span><span class="o">=</span><span class="n">buffers</span><span class="p">)</span>

  1241. <span class="nb">print</span><span class="p">(</span><span class="n">b</span> <span class="o">==</span> <span class="n">new_b</span><span class="p">)</span>  <span class="c1"># True</span>

  1242. <span class="nb">print</span><span class="p">(</span><span class="n">b</span> <span class="ow">is</span> <span class="n">new_b</span><span class="p">)</span>  <span class="c1"># True: no copy was made</span>

  1243. </pre></div>

  1244. </div>

  1245. <p>This example is limited by the fact that <a class="reference internal" href="stdtypes.html#bytearray" title="bytearray"><code class="xref py py-class docutils literal notranslate"><span class="pre">bytearray</span></code></a> allocates its

  1246. own memory: you cannot create a <a class="reference internal" href="stdtypes.html#bytearray" title="bytearray"><code class="xref py py-class docutils literal notranslate"><span class="pre">bytearray</span></code></a> instance that is backed

  1247. by another object’s memory.  However, third-party datatypes such as NumPy

  1248. arrays do not have this limitation, and allow use of zero-copy pickling

  1249. (or making as few copies as possible) when transferring between distinct

  1250. processes or systems.</p>

  1251. <div class="admonition seealso">

  1252. <p class="admonition-title">See also</p>

  1253. <p><span class="target" id="index-8"></span><a class="pep reference external" href="https://peps.python.org/pep-0574/"><strong>PEP 574</strong></a> – Pickle protocol 5 with out-of-band data</p>

  1254. </div>

  1255. </section>

  1256. </section>

  1257. <section id="restricting-globals">

  1258. <span id="pickle-restrict"></span><h2>Restricting Globals<a class="headerlink" href="#restricting-globals" title="Link to this heading">¶</a></h2>

  1259. <p id="index-9">By default, unpickling will import any class or function that it finds in the

  1260. pickle data.  For many applications, this behaviour is unacceptable as it

  1261. permits the unpickler to import and invoke arbitrary code.  Just consider what

  1262. this hand-crafted pickle data stream does when loaded:</p>

  1263. <div class="highlight-python3 notranslate"><div class="highlight"><pre><span></span><span class="gp">&gt;&gt;&gt; </span><span class="kn">import</span> <span class="nn">pickle</span>

  1264. <span class="gp">&gt;&gt;&gt; </span><span class="n">pickle</span><span class="o">.</span><span class="n">loads</span><span class="p">(</span><span class="sa">b</span><span class="s2">&quot;cos</span><span class="se">\n</span><span class="s2">system</span><span class="se">\n</span><span class="s2">(S&#39;echo hello world&#39;</span><span class="se">\n</span><span class="s2">tR.&quot;</span><span class="p">)</span>

  1265. <span class="go">hello world</span>

  1266. <span class="go">0</span>

  1267. </pre></div>

  1268. </div>

  1269. <p>In this example, the unpickler imports the <a class="reference internal" href="os.html#os.system" title="os.system"><code class="xref py py-func docutils literal notranslate"><span class="pre">os.system()</span></code></a> function and then

  1270. apply the string argument “echo hello world”.  Although this example is

  1271. inoffensive, it is not difficult to imagine one that could damage your system.</p>

  1272. <p>For this reason, you may want to control what gets unpickled by customizing

  1273. <a class="reference internal" href="#pickle.Unpickler.find_class" title="pickle.Unpickler.find_class"><code class="xref py py-meth docutils literal notranslate"><span class="pre">Unpickler.find_class()</span></code></a>.  Unlike its name suggests,

  1274. <a class="reference internal" href="#pickle.Unpickler.find_class" title="pickle.Unpickler.find_class"><code class="xref py py-meth docutils literal notranslate"><span class="pre">Unpickler.find_class()</span></code></a> is called whenever a global (i.e., a class or

  1275. a function) is requested.  Thus it is possible to either completely forbid

  1276. globals or restrict them to a safe subset.</p>

  1277. <p>Here is an example of an unpickler allowing only few safe classes from the

  1278. <a class="reference internal" href="builtins.html#module-builtins" title="builtins: The module that provides the built-in namespace."><code class="xref py py-mod docutils literal notranslate"><span class="pre">builtins</span></code></a> module to be loaded:</p>

  1279. <div class="highlight-python3 notranslate"><div class="highlight"><pre><span></span><span class="kn">import</span> <span class="nn">builtins</span>

  1280. <span class="kn">import</span> <span class="nn">io</span>

  1281. <span class="kn">import</span> <span class="nn">pickle</span>

  1282. 

  1283. <span class="n">safe_builtins</span> <span class="o">=</span> <span class="p">{</span>

  1284.     <span class="s1">&#39;range&#39;</span><span class="p">,</span>

  1285.     <span class="s1">&#39;complex&#39;</span><span class="p">,</span>

  1286.     <span class="s1">&#39;set&#39;</span><span class="p">,</span>

  1287.     <span class="s1">&#39;frozenset&#39;</span><span class="p">,</span>

  1288.     <span class="s1">&#39;slice&#39;</span><span class="p">,</span>

  1289. <span class="p">}</span>

  1290. 

  1291. <span class="k">class</span> <span class="nc">RestrictedUnpickler</span><span class="p">(</span><span class="n">pickle</span><span class="o">.</span><span class="n">Unpickler</span><span class="p">):</span>

  1292. 

  1293.     <span class="k">def</span> <span class="nf">find_class</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">module</span><span class="p">,</span> <span class="n">name</span><span class="p">):</span>

  1294.         <span class="c1"># Only allow safe classes from builtins.</span>

  1295.         <span class="k">if</span> <span class="n">module</span> <span class="o">==</span> <span class="s2">&quot;builtins&quot;</span> <span class="ow">and</span> <span class="n">name</span> <span class="ow">in</span> <span class="n">safe_builtins</span><span class="p">:</span>

  1296.             <span class="k">return</span> <span class="nb">getattr</span><span class="p">(</span><span class="n">builtins</span><span class="p">,</span> <span class="n">name</span><span class="p">)</span>

  1297.         <span class="c1"># Forbid everything else.</span>

  1298.         <span class="k">raise</span> <span class="n">pickle</span><span class="o">.</span><span class="n">UnpicklingError</span><span class="p">(</span><span class="s2">&quot;global &#39;</span><span class="si">%s</span><span class="s2">.</span><span class="si">%s</span><span class="s2">&#39; is forbidden&quot;</span> <span class="o">%</span>

  1299.                                      <span class="p">(</span><span class="n">module</span><span class="p">,</span> <span class="n">name</span><span class="p">))</span>

  1300. 

  1301. <span class="k">def</span> <span class="nf">restricted_loads</span><span class="p">(</span><span class="n">s</span><span class="p">):</span>

  1302. <span class="w">    </span><span class="sd">&quot;&quot;&quot;Helper function analogous to pickle.loads().&quot;&quot;&quot;</span>

  1303.     <span class="k">return</span> <span class="n">RestrictedUnpickler</span><span class="p">(</span><span class="n">io</span><span class="o">.</span><span class="n">BytesIO</span><span class="p">(</span><span class="n">s</span><span class="p">))</span><span class="o">.</span><span class="n">load</span><span class="p">()</span>

  1304. </pre></div>

  1305. </div>

  1306. <p>A sample usage of our unpickler working as intended:</p>

  1307. <div class="highlight-python3 notranslate"><div class="highlight"><pre><span></span><span class="gp">&gt;&gt;&gt; </span><span class="n">restricted_loads</span><span class="p">(</span><span class="n">pickle</span><span class="o">.</span><span class="n">dumps</span><span class="p">([</span><span class="mi">1</span><span class="p">,</span> <span class="mi">2</span><span class="p">,</span> <span class="nb">range</span><span class="p">(</span><span class="mi">15</span><span class="p">)]))</span>

  1308. <span class="go">[1, 2, range(0, 15)]</span>

  1309. <span class="gp">&gt;&gt;&gt; </span><span class="n">restricted_loads</span><span class="p">(</span><span class="sa">b</span><span class="s2">&quot;cos</span><span class="se">\n</span><span class="s2">system</span><span class="se">\n</span><span class="s2">(S&#39;echo hello world&#39;</span><span class="se">\n</span><span class="s2">tR.&quot;</span><span class="p">)</span>

  1310. <span class="gt">Traceback (most recent call last):</span>

  1311. <span class="w">  </span><span class="c">...</span>

  1312. <span class="gr">pickle.UnpicklingError</span>: <span class="n">global &#39;os.system&#39; is forbidden</span>

  1313. <span class="gp">&gt;&gt;&gt; </span><span class="n">restricted_loads</span><span class="p">(</span><span class="sa">b</span><span class="s1">&#39;cbuiltins</span><span class="se">\n</span><span class="s1">eval</span><span class="se">\n</span><span class="s1">&#39;</span>

  1314. <span class="gp">... </span>                 <span class="sa">b</span><span class="s1">&#39;(S</span><span class="se">\&#39;</span><span class="s1">getattr(__import__(&quot;os&quot;), &quot;system&quot;)&#39;</span>

  1315. <span class="gp">... </span>                 <span class="sa">b</span><span class="s1">&#39;(&quot;echo hello world&quot;)</span><span class="se">\&#39;\n</span><span class="s1">tR.&#39;</span><span class="p">)</span>

  1316. <span class="gt">Traceback (most recent call last):</span>

  1317. <span class="w">  </span><span class="c">...</span>

  1318. <span class="gr">pickle.UnpicklingError</span>: <span class="n">global &#39;builtins.eval&#39; is forbidden</span>

  1319. </pre></div>

  1320. </div>

  1321. <p>As our examples shows, you have to be careful with what you allow to be

  1322. unpickled.  Therefore if security is a concern, you may want to consider

  1323. alternatives such as the marshalling API in <a class="reference internal" href="xmlrpc.client.html#module-xmlrpc.client" title="xmlrpc.client: XML-RPC client access."><code class="xref py py-mod docutils literal notranslate"><span class="pre">xmlrpc.client</span></code></a> or

  1324. third-party solutions.</p>

  1325. </section>

  1326. <section id="performance">

  1327. <h2>Performance<a class="headerlink" href="#performance" title="Link to this heading">¶</a></h2>

  1328. <p>Recent versions of the pickle protocol (from protocol 2 and upwards) feature

  1329. efficient binary encodings for several common features and built-in types.

  1330. Also, the <a class="reference internal" href="#module-pickle" title="pickle: Convert Python objects to streams of bytes and back."><code class="xref py py-mod docutils literal notranslate"><span class="pre">pickle</span></code></a> module has a transparent optimizer written in C.</p>

  1331. </section>

  1332. <section id="examples">

  1333. <span id="pickle-example"></span><h2>Examples<a class="headerlink" href="#examples" title="Link to this heading">¶</a></h2>

  1334. <p>For the simplest code, use the <a class="reference internal" href="#pickle.dump" title="pickle.dump"><code class="xref py py-func docutils literal notranslate"><span class="pre">dump()</span></code></a> and <a class="reference internal" href="#pickle.load" title="pickle.load"><code class="xref py py-func docutils literal notranslate"><span class="pre">load()</span></code></a> functions.</p>

  1335. <div class="highlight-python3 notranslate"><div class="highlight"><pre><span></span><span class="kn">import</span> <span class="nn">pickle</span>

  1336. 

  1337. <span class="c1"># An arbitrary collection of objects supported by pickle.</span>

  1338. <span class="n">data</span> <span class="o">=</span> <span class="p">{</span>

  1339.     <span class="s1">&#39;a&#39;</span><span class="p">:</span> <span class="p">[</span><span class="mi">1</span><span class="p">,</span> <span class="mf">2.0</span><span class="p">,</span> <span class="mi">3</span><span class="o">+</span><span class="mi">4</span><span class="n">j</span><span class="p">],</span>

  1340.     <span class="s1">&#39;b&#39;</span><span class="p">:</span> <span class="p">(</span><span class="s2">&quot;character string&quot;</span><span class="p">,</span> <span class="sa">b</span><span class="s2">&quot;byte string&quot;</span><span class="p">),</span>

  1341.     <span class="s1">&#39;c&#39;</span><span class="p">:</span> <span class="p">{</span><span class="kc">None</span><span class="p">,</span> <span class="kc">True</span><span class="p">,</span> <span class="kc">False</span><span class="p">}</span>

  1342. <span class="p">}</span>

  1343. 

  1344. <span class="k">with</span> <span class="nb">open</span><span class="p">(</span><span class="s1">&#39;data.pickle&#39;</span><span class="p">,</span> <span class="s1">&#39;wb&#39;</span><span class="p">)</span> <span class="k">as</span> <span class="n">f</span><span class="p">:</span>

  1345.     <span class="c1"># Pickle the &#39;data&#39; dictionary using the highest protocol available.</span>

  1346.     <span class="n">pickle</span><span class="o">.</span><span class="n">dump</span><span class="p">(</span><span class="n">data</span><span class="p">,</span> <span class="n">f</span><span class="p">,</span> <span class="n">pickle</span><span class="o">.</span><span class="n">HIGHEST_PROTOCOL</span><span class="p">)</span>

  1347. </pre></div>

  1348. </div>

  1349. <p>The following example reads the resulting pickled data.</p>

  1350. <div class="highlight-python3 notranslate"><div class="highlight"><pre><span></span><span class="kn">import</span> <span class="nn">pickle</span>

  1351. 

  1352. <span class="k">with</span> <span class="nb">open</span><span class="p">(</span><span class="s1">&#39;data.pickle&#39;</span><span class="p">,</span> <span class="s1">&#39;rb&#39;</span><span class="p">)</span> <span class="k">as</span> <span class="n">f</span><span class="p">:</span>

  1353.     <span class="c1"># The protocol version used is detected automatically, so we do not</span>

  1354.     <span class="c1"># have to specify it.</span>

  1355.     <span class="n">data</span> <span class="o">=</span> <span class="n">pickle</span><span class="o">.</span><span class="n">load</span><span class="p">(</span><span class="n">f</span><span class="p">)</span>

  1356. </pre></div>

  1357. </div>

  1358. <div class="admonition seealso">

  1359. <p class="admonition-title">See also</p>

  1360. <dl class="simple">

  1361. <dt>Module <a class="reference internal" href="copyreg.html#module-copyreg" title="copyreg: Register pickle support functions."><code class="xref py py-mod docutils literal notranslate"><span class="pre">copyreg</span></code></a></dt><dd><p>Pickle interface constructor registration for extension types.</p>

  1362. </dd>

  1363. <dt>Module <a class="reference internal" href="pickletools.html#module-pickletools" title="pickletools: Contains extensive comments about the pickle protocols and pickle-machine opcodes, as well as some useful functions."><code class="xref py py-mod docutils literal notranslate"><span class="pre">pickletools</span></code></a></dt><dd><p>Tools for working with and analyzing pickled data.</p>

  1364. </dd>

  1365. <dt>Module <a class="reference internal" href="shelve.html#module-shelve" title="shelve: Python object persistence."><code class="xref py py-mod docutils literal notranslate"><span class="pre">shelve</span></code></a></dt><dd><p>Indexed databases of objects; uses <a class="reference internal" href="#module-pickle" title="pickle: Convert Python objects to streams of bytes and back."><code class="xref py py-mod docutils literal notranslate"><span class="pre">pickle</span></code></a>.</p>

  1366. </dd>

  1367. <dt>Module <a class="reference internal" href="copy.html#module-copy" title="copy: Shallow and deep copy operations."><code class="xref py py-mod docutils literal notranslate"><span class="pre">copy</span></code></a></dt><dd><p>Shallow and deep object copying.</p>

  1368. </dd>

  1369. <dt>Module <a class="reference internal" href="marshal.html#module-marshal" title="marshal: Convert Python objects to streams of bytes and back (with different constraints)."><code class="xref py py-mod docutils literal notranslate"><span class="pre">marshal</span></code></a></dt><dd><p>High-performance serialization of built-in types.</p>

  1370. </dd>

  1371. </dl>

  1372. </div>

  1373. <p class="rubric">Footnotes</p>

  1374. <aside class="footnote-list brackets">

  1375. <aside class="footnote brackets" id="id7" role="doc-footnote">

  1376. <span class="label"><span class="fn-bracket">[</span><a role="doc-backlink" href="#id1">1</a><span class="fn-bracket">]</span></span>

  1377. <p>Don’t confuse this with the <a class="reference internal" href="marshal.html#module-marshal" title="marshal: Convert Python objects to streams of bytes and back (with different constraints)."><code class="xref py py-mod docutils literal notranslate"><span class="pre">marshal</span></code></a> module</p>

  1378. </aside>

  1379. <aside class="footnote brackets" id="id8" role="doc-footnote">

  1380. <span class="label"><span class="fn-bracket">[</span><a role="doc-backlink" href="#id3">2</a><span class="fn-bracket">]</span></span>

  1381. <p>This is why <a class="reference internal" href="../reference/expressions.html#lambda"><code class="xref std std-keyword docutils literal notranslate"><span class="pre">lambda</span></code></a> functions cannot be pickled:  all

  1382. <code class="xref std std-keyword docutils literal notranslate"><span class="pre">lambda</span></code> functions share the same name:  <code class="docutils literal notranslate"><span class="pre">&lt;lambda&gt;</span></code>.</p>

  1383. </aside>

  1384. <aside class="footnote brackets" id="id9" role="doc-footnote">

  1385. <span class="label"><span class="fn-bracket">[</span><a role="doc-backlink" href="#id4">3</a><span class="fn-bracket">]</span></span>

  1386. <p>The exception raised will likely be an <a class="reference internal" href="exceptions.html#ImportError" title="ImportError"><code class="xref py py-exc docutils literal notranslate"><span class="pre">ImportError</span></code></a> or an

  1387. <a class="reference internal" href="exceptions.html#AttributeError" title="AttributeError"><code class="xref py py-exc docutils literal notranslate"><span class="pre">AttributeError</span></code></a> but it could be something else.</p>

  1388. </aside>

  1389. <aside class="footnote brackets" id="id10" role="doc-footnote">

  1390. <span class="label"><span class="fn-bracket">[</span><a role="doc-backlink" href="#id5">4</a><span class="fn-bracket">]</span></span>

  1391. <p>The <a class="reference internal" href="copy.html#module-copy" title="copy: Shallow and deep copy operations."><code class="xref py py-mod docutils literal notranslate"><span class="pre">copy</span></code></a> module uses this protocol for shallow and deep copying

  1392. operations.</p>

  1393. </aside>

  1394. <aside class="footnote brackets" id="id11" role="doc-footnote">

  1395. <span class="label"><span class="fn-bracket">[</span><a role="doc-backlink" href="#id6">5</a><span class="fn-bracket">]</span></span>

  1396. <p>The limitation on alphanumeric characters is due to the fact

  1397. that persistent IDs in protocol 0 are delimited by the newline

  1398. character.  Therefore if any kind of newline characters occurs in

  1399. persistent IDs, the resulting pickled data will become unreadable.</p>

  1400. </aside>

  1401. </aside>

  1402. </section>

  1403. </section>

  1404. 

  1405. 

  1406.             <div class="clearer"></div>

  1407.           </div>

  1408.         </div>

  1409.       </div>

  1410.       <div class="sphinxsidebar" role="navigation" aria-label="main navigation">

  1411.         <div class="sphinxsidebarwrapper">

  1412.   <div>

  1413.     <h3><a href="../contents.html">Table of Contents</a></h3>

  1414.     <ul>

  1415. <li><a class="reference internal" href="#"><code class="xref py py-mod docutils literal notranslate"><span class="pre">pickle</span></code> — Python object serialization</a><ul>

  1416. <li><a class="reference internal" href="#relationship-to-other-python-modules">Relationship to other Python modules</a><ul>

  1417. <li><a class="reference internal" href="#comparison-with-marshal">Comparison with <code class="docutils literal notranslate"><span class="pre">marshal</span></code></a></li>

  1418. <li><a class="reference internal" href="#comparison-with-json">Comparison with <code class="docutils literal notranslate"><span class="pre">json</span></code></a></li>

  1419. </ul>

  1420. </li>

  1421. <li><a class="reference internal" href="#data-stream-format">Data stream format</a></li>

  1422. <li><a class="reference internal" href="#module-interface">Module Interface</a></li>

  1423. <li><a class="reference internal" href="#what-can-be-pickled-and-unpickled">What can be pickled and unpickled?</a></li>

  1424. <li><a class="reference internal" href="#pickling-class-instances">Pickling Class Instances</a><ul>

  1425. <li><a class="reference internal" href="#persistence-of-external-objects">Persistence of External Objects</a></li>

  1426. <li><a class="reference internal" href="#dispatch-tables">Dispatch Tables</a></li>

  1427. <li><a class="reference internal" href="#handling-stateful-objects">Handling Stateful Objects</a></li>

  1428. </ul>

  1429. </li>

  1430. <li><a class="reference internal" href="#custom-reduction-for-types-functions-and-other-objects">Custom Reduction for Types, Functions, and Other Objects</a></li>

  1431. <li><a class="reference internal" href="#out-of-band-buffers">Out-of-band Buffers</a><ul>

  1432. <li><a class="reference internal" href="#provider-api">Provider API</a></li>

  1433. <li><a class="reference internal" href="#consumer-api">Consumer API</a></li>

  1434. <li><a class="reference internal" href="#example">Example</a></li>

  1435. </ul>

  1436. </li>

  1437. <li><a class="reference internal" href="#restricting-globals">Restricting Globals</a></li>

  1438. <li><a class="reference internal" href="#performance">Performance</a></li>

  1439. <li><a class="reference internal" href="#examples">Examples</a></li>

  1440. </ul>

  1441. </li>

  1442. </ul>

  1443. 

  1444.   </div>

  1445.   <div>

  1446.     <h4>Previous topic</h4>

  1447.     <p class="topless"><a href="persistence.html"

  1448.                           title="previous chapter">Data Persistence</a></p>

  1449.   </div>

  1450.   <div>

  1451.     <h4>Next topic</h4>

  1452.     <p class="topless"><a href="copyreg.html"

  1453.                           title="next chapter"><code class="xref py py-mod docutils literal notranslate"><span class="pre">copyreg</span></code> — Register <code class="xref py py-mod docutils literal notranslate"><span class="pre">pickle</span></code> support functions</a></p>

  1454.   </div>

  1455.   <div role="note" aria-label="source link">

  1456.     <h3>This Page</h3>

  1457.     <ul class="this-page-menu">

  1458.       <li><a href="../bugs.html">Report a Bug</a></li>

  1459.       <li>

  1460.         <a href="https://github.com/python/cpython/blob/main/Doc/library/pickle.rst"

  1461.             rel="nofollow">Show Source

  1462.         </a>

  1463.       </li>

  1464.     </ul>

  1465.   </div>

  1466.         </div>

  1467. <div id="sidebarbutton" title="Collapse sidebar">

  1468. <span>«</span>

  1469. </div>

  1470. 

  1471.       </div>

  1472.       <div class="clearer"></div>

  1473.     </div>  

  1474.     <div class="related" role="navigation" aria-label="related navigation">

  1475.       <h3>Navigation</h3>

  1476.       <ul>

  1477.         <li class="right" style="margin-right: 10px">

  1478.           <a href="../genindex.html" title="General Index"

  1479.              >index</a></li>

  1480.         <li class="right" >

  1481.           <a href="../py-modindex.html" title="Python Module Index"

  1482.              >modules</a> |</li>

  1483.         <li class="right" >

  1484.           <a href="copyreg.html" title="copyreg — Register pickle support functions"

  1485.              >next</a> |</li>

  1486.         <li class="right" >

  1487.           <a href="persistence.html" title="Data Persistence"

  1488.              >previous</a> |</li>

  1489. 

  1490.           <li><img src="../_static/py.svg" alt="Python logo" style="vertical-align: middle; margin-top: -1px"/></li>

  1491.           <li><a href="https://www.python.org/">Python</a> &#187;</li>

  1492.           <li class="switchers">

  1493.             <div class="language_switcher_placeholder"></div>

  1494.             <div class="version_switcher_placeholder"></div>

  1495.           </li>

  1496.           <li>

  1497.               

  1498.           </li>

  1499.     <li id="cpython-language-and-version">

  1500.       <a href="../index.html">3.12.3 Documentation</a> &#187;

  1501.     </li>

  1502. 

  1503.           <li class="nav-item nav-item-1"><a href="index.html" >The Python Standard Library</a> &#187;</li>

  1504.           <li class="nav-item nav-item-2"><a href="persistence.html" >Data Persistence</a> &#187;</li>

  1505.         <li class="nav-item nav-item-this"><a href=""><code class="xref py py-mod docutils literal notranslate"><span class="pre">pickle</span></code> — Python object serialization</a></li>

  1506.                 <li class="right">

  1507.                     

  1508. 

  1509.     <div class="inline-search" role="search">

  1510.         <form class="inline-search" action="../search.html" method="get">

  1511.           <input placeholder="Quick search" aria-label="Quick search" type="search" name="q" id="search-box" />

  1512.           <input type="submit" value="Go" />

  1513.         </form>

  1514.     </div>

  1515.                      |

  1516.                 </li>

  1517.             <li class="right">

  1518. <label class="theme-selector-label">

  1519.     Theme

  1520.     <select class="theme-selector" oninput="activateTheme(this.value)">

  1521.         <option value="auto" selected>Auto</option>

  1522.         <option value="light">Light</option>

  1523.         <option value="dark">Dark</option>

  1524.     </select>

  1525. </label> |</li>

  1526.             

  1527.       </ul>

  1528.     </div>  

  1529.     <div class="footer">

  1530.     &copy; 

  1531.       <a href="../copyright.html">

  1532.     

  1533.     Copyright

  1534.     

  1535.       </a>

  1536.      2001-2024, Python Software Foundation.

  1537.     <br />

  1538.     This page is licensed under the Python Software Foundation License Version 2.

  1539.     <br />

  1540.     Examples, recipes, and other code in the documentation are additionally licensed under the Zero Clause BSD License.

  1541.     <br />

  1542.     

  1543.       See <a href="/license.html">History and License</a> for more information.<br />

  1544.     

  1545.     

  1546.     <br />

  1547. 

  1548.     The Python Software Foundation is a non-profit corporation.

  1549. <a href="https://www.python.org/psf/donations/">Please donate.</a>

  1550. <br />

  1551.     <br />

  1552.       Last updated on Apr 09, 2024 (13:47 UTC).

  1553.     

  1554.       <a href="/bugs.html">Found a bug</a>?

  1555.     

  1556.     <br />

  1557. 

  1558.     Created using <a href="https://www.sphinx-doc.org/">Sphinx</a> 7.2.6.

  1559.     </div>

  1560. 

  1561.   </body>

  1562. </html>
上海开阖软件有限公司 沪ICP备12045867号-1