| 123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122 |
- # multiprocessing
- John Melesky
- (PDX Python, June 2010)
- ---
- # In the 2.6 standard library
- Otherwise, available from PyPI (>=2.4)
- ---
- # A note
- `multiprocessing` is a huge library, and deserving of a full-fledged talk (or several), rather than a module-of-the-month.
- The manager functionality, for example, is elaborate and useful.
- ---
- # A note
- `multiprocessing` is a huge library, and deserving of a full-fledged talk (or several), rather than a module-of-the-month.
- The manager functionality, for example, is elaborate and useful (as far as i can tell. i haven't used it. this is still just a motm talk).
- ---
- # A different note
- `multiprocessing` is complex, and occasionally slightly hackish. Expect the following:
- - These examples may not work in the REPL
- - Similar arguments may have different semantics depending on whether they're passed to `Process` or `Pool`
- - It "works" on Windows
- - Overall, YMMV
- ---
- # Pre-multiprocessing
- - fork/exec
- - threading
- ---
- # Pre-multiprocessing
- - fork/exec
- - threading (inspired by Java!)
- ---
- # Process
- [sample process code](code/proc1.py)
- ---
- # Process
- - name, pid, exitcode, authkey
- - `start()`
- - `run()`
- - `join([timeout])`
- - `is_alive()`
- - `terminate()`
- - daemon
- ---
- # Pool
- [sample pool code](code/proc2.py)
- ---
- # Pool
- - `apply(func[, args[, kwds]])`
- - `apply_async(ditto, plus callback)`
- - `map`, `map_async`, `imap`, `imap_unordered`
- - `close()`
- - `terminate()`
- - `join()`
- ---
- # Queue
- [(nonworking) sample queue code](code/proc3.py)
- ---
- # Queue
- See Queue.Queue
- - `empty()`, `full()`, `qsize()`
- - `put(item[, block[, timeout]])`
- - `put_nowait(item)`
- - `get([block[, timeout]])`
- - `get_nowait()`
- ---
- # Pipe
- [(semiworking) sample pipe code](code/proc4.py)
- ---
- # Connection
- - `send(obj)`
- - `recv()`
- - `fileno()`
- - `close()`
- - `poll([timeout])`
- - `send_bytes`, `recv_bytes`, `recv_bytes_into`
- ---
- # Stuff i'm not covering
- - `Lock`
- - shared memory maps (`Value`, `Array`)
- - `Manager`s, both local and remote
- - Proxies
- - `Namespace`
- - `Listener` and `Client`
- - lots of other stuff
- ---
- # Final example
- A [web scraper](code/scraper.py) with Process and Queue
- ---
|
