Kausta

joined 1 year ago
[โ€“] Kausta@lemm.ee 9 points 3 months ago

Yeah, it was just a simple example. Although using just pandas (without something like dask) for loading terabytes of data at once into a single dataframe may not be the best idea, even with enough memory.

[โ€“] Kausta@lemm.ee 66 points 3 months ago (4 children)

You havent seen anything until you need to put a 4.2gb gzipped csv into a pandas dataframe, which works without any issues I should note.