Call multiple C ++ functions in python using threads
Suppose I have a C (++) function taking an integer, and it is linked to (p) python with python api, so I can call it from python:
import c_module
c_module.f(10)
now, i want to parallelize it. The problem is, how does the GIL work in this case? Suppose I have a queue of numbers to process and some workers ( threading.Thread
) are running in parallel, each calling c_module.f(number)
where it number
is taken from the queue.
The difference with the usual case where the GIL blocks an interpreter is that now you don't need an interpreter to evaluate c_module.f
, because it is compiled. So the question is, is the processing really parallel in this case?
a source to share
Themes currently executed by C extension code for which the GIL has been explicitly released will run in parallel. See http://docs.python.org/c-api/init.html#thread-state-and-the-global-interpreter-lock for what you need to do in your extension.
Python threads are most useful for performing I / O binding or for responding to a GUI. I wouldn't do python-heavy execution with threads. If you want to guarantee parallelism check out the library multiprocessing
.
a source to share