Uploaded image for project: 'MidPoint'
  1. MidPoint
  2. MID-6145

Custom Function OperationResult Memory Leak

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: 4.0.2
    • Fix Version/s: 4.1
    • Component/s: None
    • Labels:
      None
    • Environment:

      Debian 10, OpenJDK-11, PostgreSQL

    • Subscription:
      No subscription (community)
    • Backport Version:

      Description

      This ticket is a continuation of a message I sent to the mailing list:
       


      I've noticed since moving onto Midpoint 4.0.2 from 3.9 that my logs have been filled with warnings such as this:
       __ 
      2020-03-18 14:23:38,491 [MODEL] [pool-3-thread-7] WARN (com.evolveum.midpoint.model.common.expression.functions.CustomFunctions): No ScriptExpressionEvaluationContext for current thread found. Using initialization-time task and operation result: Task(id:1565719367356-0-1, name:TestCaseReconcile, oid:27ea7035-a8db-42cd-b2a2-cb997b30e6f9)
       __ 
      At the same time I have also noticed that while large tasks are being executed the heap slowly fills up and eventually Midpoint becomes unresponsive. I took a heap dump and found a massive OperationResult object retaining over 90% of my 14GB heap. Forcing garbage collection with something like VisualVM doesn't seem to free the memory, even after the task is done running. 
       __ 
      My assumption is that the giant OperationResult is made up of whatever data is being appended to it by the CustomFunctions executions. I've traced the CustomFunctions class and the warning seems intermittent, in the same thread sometimes for a given function it'll throw the warning, sometimes it won't.


       
      Since I sent it I've been able to troubleshoot it a little bit more. The issue is still intermittent and it's not exactly clear what's causing it in my environment. I've been one-by-one examining calls to custom functions in my business logic and trying to resolve individual instances of the warning; which has been hit-or-miss.
       
      So far I've found one reliable way to trigger the warning and leak memory, which is repeatedly calling a custom function. I wrote a collection of objects that when imported into a clean midPoint 4.0.2 illustrates the bug (it's more or less just a single call in a huge for loop). This is the attached XML file.
       
      In my environment instead of repeatedly calling the same function in one mapping, I have many separate mappings calling a custom function very similar to the one I've included in the sample. 
       
      Pavol Mederly responded to my message asking what happened if I set workerThreads to 0. I did this in the task that I originally saw this issue and it doesn't appear to have had any effect, I'm still flooded by the warnings.
       

        Attachments

          Issue Links

            Activity

              People

              Assignee:
              fbc216 Forest Crowley
              Reporter:
              fbc216 Forest Crowley
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved: