Home Why Virtual Memory grows during map_async operation with python?
Reply: 0

Why Virtual Memory grows during map_async operation with python?

user1003
1#
user1003 Published in April 19, 2018, 9:54 am

I am trying to create interatomic distances of N=6000 atoms for 100 frames. Each frame contains atom positions in a periodic boundary box. I am trying to parallelize this process using multiprocessing tools in python. Mainly I use map_async function:

    q=Pool(np)
    for group in groups:
    first_frames.append(group[0])
    start1=timer()
    #s=q.map(dipoles.new_create_neighborlist,first_frames)
    s=q.map_async(dipoles.new_create_neighborlist,first_frames)
    q.close()
    q.join()

first_frames is a list of objects which contains atom positions etc. Length of the first_frames list is 1000. Number of processors is 16. It starts well and does the computation without problem. However after some time virtual memory used by the processes grows and gives virtual memory error. dipoles.new_create_neighborlist returns a list of atoms with its neighborlists. I tried previously answered methods but did not work. What could I be missing here?

You need to login account before you can post.

About| Privacy statement| Terms of Service| Advertising| Contact us| Help| Sitemap|
Processed in 0.301205 second(s) , Gzip On .

© 2016 Powered by mzan.com design MATCHINFO