jmelesky
/
pdorg_site


			
							123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199200201202203204205206207208209210211212213214215216217218219220221222223224225226227228229230231232233234235
							<html> <head>
<title>multiprocessing as the module of the month</title>

<style>

body {
  background-color: #cfdada;
}

.slide {
  font-family: Gill Sans;
  border: 2px solid #778888;
  background-color: #e7fafa;;
  padding: 2%;
  width: 94%;
}

h1 {
  text-align: center;
}

pre {
  border: 1px solid #446666;
  background-color: #e8ffff;
  padding: 2px;
}


</style>

<script src="scripts/jquery-1.2.3.js" type="text/javascript"></script>
<script src="scripts/slideshow.js" type="text/javascript"></script>
</head>

<body>

<div class='slide'>
<h1>multiprocessing</h1>

<p>John Melesky
(PDX Python, June 2010)</p>

</div>


<div class='slide'>
<h1>In the 2.6 standard library</h1>

<p>Otherwise, available from PyPI (&gt;=2.4)</p>

</div>


<div class='slide'>
<h1>A note</h1>

<p><code>multiprocessing</code> is a huge library, and deserving of a full-fledged talk (or several), rather than a module-of-the-month.</p>

<p>The manager functionality, for example, is elaborate and useful.</p>

</div>


<div class='slide'>
<h1>A note</h1>

<p><code>multiprocessing</code> is a huge library, and deserving of a full-fledged talk (or several), rather than a module-of-the-month.</p>

<p>The manager functionality, for example, is elaborate and useful (as far as i can tell. i haven't used it. this is still just a motm talk).</p>

</div>


<div class='slide'>
<h1>A different note</h1>

<p><code>multiprocessing</code> is complex, and occasionally slightly hackish. Expect the following:</p>

<ul>
<li>These examples may not work in the REPL</li>
<li>Similar arguments may have different semantics depending on whether they're passed to <code>Process</code> or <code>Pool</code></li>
<li>It "works" on Windows</li>
<li>Overall, YMMV</li>
</ul>

</div>


<div class='slide'>
<h1>Pre-multiprocessing</h1>

<ul>
<li>fork/exec</li>
<li>threading</li>
</ul>

</div>


<div class='slide'>
<h1>Pre-multiprocessing</h1>

<ul>
<li>fork/exec</li>
<li>threading (inspired by Java!)</li>
</ul>

</div>


<div class='slide'>
<h1>Process</h1>

<p><a href="code/proc1.py">sample process code</a></p>

</div>


<div class='slide'>
<h1>Process</h1>

<ul>
<li>name, pid, exitcode, authkey</li>
<li><code>start()</code></li>
<li><code>run()</code></li>
<li><code>join([timeout])</code></li>
<li><code>is_alive()</code></li>
<li><code>terminate()</code></li>
<li>daemon</li>
</ul>

</div>


<div class='slide'>
<h1>Pool</h1>

<p><a href="code/proc2.py">sample pool code</a></p>

</div>


<div class='slide'>
<h1>Pool</h1>

<ul>
<li><code>apply(func[, args[, kwds]])</code></li>
<li><code>apply_async(ditto, plus callback)</code></li>
<li><code>map</code>, <code>map_async</code>, <code>imap</code>, <code>imap_unordered</code></li>
<li><code>close()</code></li>
<li><code>terminate()</code></li>
<li><code>join()</code></li>
</ul>

</div>


<div class='slide'>
<h1>Queue</h1>

<p><a href="code/proc3.py">(nonworking) sample queue code</a></p>

</div>


<div class='slide'>
<h1>Queue</h1>

<p>See Queue.Queue</p>

<ul>
<li><code>empty()</code>, <code>full()</code>, <code>qsize()</code></li>
<li><code>put(item[, block[, timeout]])</code></li>
<li><code>put_nowait(item)</code></li>
<li><code>get([block[, timeout]])</code></li>
<li><code>get_nowait()</code></li>
</ul>

</div>


<div class='slide'>
<h1>Pipe</h1>

<p><a href="code/proc4.py">(semiworking) sample pipe code</a></p>

</div>


<div class='slide'>
<h1>Connection</h1>

<ul>
<li><code>send(obj)</code></li>
<li><code>recv()</code></li>
<li><code>fileno()</code></li>
<li><code>close()</code></li>
<li><code>poll([timeout])</code></li>
<li><code>send_bytes</code>, <code>recv_bytes</code>, <code>recv_bytes_into</code></li>
</ul>

</div>


<div class='slide'>
<h1>Stuff i'm not covering</h1>

<ul>
<li><code>Lock</code></li>
<li>shared memory maps (<code>Value</code>, <code>Array</code>)</li>
<li><code>Manager</code>s, both local and remote</li>
<li>Proxies</li>
<li><code>Namespace</code></li>
<li><code>Listener</code> and <code>Client</code></li>
<li>lots of other stuff</li>
</ul>

</div>


<div class='slide'>
<h1>Final example</h1>

<p>A <a href="code/scraper.py">web scraper</a> with Process and Queue</p>

</div>


<div class='slide'>
<p></p>

</div>


</body></html>