Background:
I have been using AutoIntegrate for several years and have processed well over 100 images. Many of my images are over 20 hours of 300 sec exposures and take several hours to process.
My computer is an older Dell i9 8th Gen. It only has 32GB memory. Paged pool 1.2Gb, Non paged 1.1 Gb. Solid State hard drive with about 1T free. Win 11.
I try to keep everything up to date and recently checked PI for updates and found the v1.7 and a couple others available. I did update and all went well.
Now everything seems to run fine to a point. Frequently about 2 to 4 hours into things it stops running with a out of memory error. I cannot process any image, all fail. A couple time it even crashed windows forcing me to power down and restart. Most of the time it just stops running. I tried auto continue with no luck. I tried running it with no options set (other than RCB with Ha and L option on. I tried to run with Fast Integrate checked and it ran about 10 min and failed.
Do you have any suggestions on how I can get it running again?
Unfortunately I do not have any good ideas. I do not remember any big changes in the script that could cause such problems.
I have pushed a test version to my debug repository. It uses “hard” garbage collection. If you want, you can try that. My debug repository is at address: https://ruuth.xyz/debug/autointegrate/
Maybe some PixInsight setting could help too. For example limiting the number of threads could help.
Good Morning. The 64GB memory arrived this morning and I installed it in computer. That all works.
I then tried my current project with about 22 hours total of Ha, L, R, G, and B. only option set was fast integrate and Ha, L in the RGB settings.
It ran through the Ha, L, and R fast integrations and failed during the G. I watched the memory usage and it never got over 50%.
Then I ran an old project with only about 1.5 hours of RGB. That ran fine and I think the output was better that the older version of autointegrate some time back. So maybe some disk space is getting filled up on the larger projects. I have all my astro images on a 2T external SSD which is about 1/2 full so I dont think there is any issues with that. Are working files stored any other place that might be causing the problem. My drive C is a 1T SSD and it is also about 1/2 full.
I see all the autointegrate created files on the external SSD in the folder with the source files so I know they are going where I expect.
When you say fast integration failed, did it run out of memory or failed for some other reason?
Temporary output files go to AutoOutput directory that is under the light files. I do not remember that there is an option to change that. But you can always delete all files in AutoOutput since they are not needed after processing. Processed files go to AutoProcessed directory. PixInsight may generate temporary files and I believe in Windows they go somewhere in C drive.
You have out of memory problem too? Or something else?
Do you get AutoIntegrate.log file? It might help to understand the problem, like is it the same step that always causes the problem. Also a screen output (if possible) could be useful.
Ok, I did some testing and noticed a big difference in peak memory usage between 1.69 and 1.70. So it looks like I can reproduce the issue and can try to find a fix.
The last release seems to have fixed the memory problem
If I run simple jobs (few options selected) it is working fine.
I have been on the road the last week and have not done much
I shot 154 images 2 nights ago of LRGB of M31. If I run it without drizzle it runs good (options: crop, fast integrate, noiseX, blurX). However if I add drizzle it failed.(not memory error)
Today I will try it again with drizzle and get the exact options and text back the results
Thanks
Gary
If the latest problem is AutoIntegrate problem, I am interested in checking that. I made some drizzle changes just recently. It would be very useful if you could send me the AutoIntegrate.log file (info@astroimagetools.net).
I agree that the memory problem is fixed.
I will send files but first:
I have never used the “fast integrate” option. When the memory issue popped up I as looking at everything and read a bit about it and started checking it every time. after the memory thing I kept doing it. Today it downed on me to check it so I ran a test. With the same data in clean folders —-
1- no fast integrate and drizzle
2- fast integrate and drizzle
3- fast integrate and no drizzle
#1 worked fine and #2 failed. #3 worked fine.
The data was a few hours of Ha, L, RGB ON M31.
Gary