this post was submitted on 14 Oct 2024
18 points (100.0% liked)

Python

6392 readers
36 users here now

Welcome to the Python community on the programming.dev Lemmy instance!

๐Ÿ“… Events

PastNovember 2023

October 2023

July 2023

August 2023

September 2023

๐Ÿ Python project:
๐Ÿ’“ Python Community:
โœจ Python Ecosystem:
๐ŸŒŒ Fediverse
Communities
Projects
Feeds

founded 1 year ago
MODERATORS
jnovinger@programming.dev
erlingur@programming.dev
norambna@programming.dev
18
Multiple process or threads? And, why? (programming.dev)
submitted 1 month ago by anzo@programming.dev to c/python@programming.dev
16 comments fedilink hide all child comments
 

(For context, I'm basically referring to Python 3.12 "multiprocessing.Pool Vs. concurrent.futures.ThreadPoolExecutor"...)

Today I read that multiple cores (parallelism) help in CPU bound operations. Meanwhile, multiple threads (concurrency) is due when the tasks are I/O bound.

Is this correct? Anyone cares to elaborate for me?

At least from a theorethical standpoint. Of course, many real work has a mix of both, and I'd better start with profiling where the bottlenecks really are.

If serves of anything having a concrete "algorithm". Let's say, I have a function that applies a map-reduce strategy reading data chunks from a file on disk, and I'm computing some averages from these data, and saving to a new file.

you are viewing a single comment's thread
view the rest of the comments
[โ€“] dwt 1 points 1 month ago (1 children)

On python, because of the Gil, multi processing should always be preferred if possible.

[โ€“] logging_strict@programming.dev 1 points 3 weeks ago

Also logging is not isolated. Bleeds all over the place. Which is a deal breaker

Not worth the endless time doing forensics

Agree! Lets stick with multiprocessing

one thread sounds nice. Lets do much more of that