[ENH] Improve the DropDownOptionsProviderBase with dynamic datas

[jeanbaptistedalle]

Enhancement Request

The Activity Descriptor API list all the activities but it can take too much time to respond when the activities have selects based on external data.

Enhancement Overview

For our business cases, we created numerous activities that depend on dynamic data provided in select fields, allowing users to choose from options instead of using text fields, which are prone to errors.

This dynamic data comes from external APIs and varies depending on the current tenant. To provide some context, we have about 30 activities with a total of 50 select fields that rely on dynamic data.

After upgrading to version 3.3.0-rc4, we encountered a performance issue when loading the workflow definition page. The Activity Descriptor API loads all activities and their respective fields, including those with external API dependencies. Despite implementing caching for select fields with common data, the overall process still takes around 10 seconds to prepare the data.

Consequently, every time the workflow definition page is opened, it takes an additional 10 seconds to load the list of activities. While the descriptor API isn’t called again within the same browser tab, opening a new tab triggers the API, resulting in another 10-second delay.

Alternative Solutions

We attempted an alternative implementation of DropDownOptionsProviderBase by setting the "Refresh" parameter to true, forcing the inputs to refresh when an activity is focused. In this implementation, we added a rule to make API calls only if the context is not null. This approach ensures that API calls are made only when an activity is focused, avoiding unnecessary external API calls when the Activity Descriptor API is triggered on the workflow definition page.

While this approach worked to some extent, we faced another issue: for activities with multiple select fields, some fields remained empty because they weren’t refreshed. We suspect this is due to the debouncer in the input tabs, which prevents multiple reloads. (Reference: https://github.com/elsa-workflows/elsa-studio/blob/fbb377df971cedd6e8f8e76b25f4c7ac4853474c/src/modules/Elsa.Studio.Workflows/Components/WorkflowDefinitionEditor/Components/ActivityProperties/Tabs/InputsTab.razor.cs#L31C1-L32C1).

As a temporary workaround, we added a Task.Delay in our implementation of DropDownOptionsProvider, which seems to resolve the issue by preventing the debouncer from canceling some refresh operations. However, we would prefer a cleaner and more robust fix for this case.

Proposed Enhancement

This is an open topic, as it could impact both elsa-core and elsa-studio, depending on the chosen solution.

We are unsure of the debouncer’s specific purpose, so we are hesitant to suggest its removal. Perhaps a new refresh mechanism could be implemented specifically for dynamic data, one that is not affected by the debouncer?

Use Cases

This enhancement would significantly improve loading times for implementations of Elsa Workflows that heavily rely on external data for their inputs.

[jdevillard]

Thanks for reporting,

As I see, about the debouncer, I think it's necessary because (in my mind) the call is made not only when you focus on the activity, but also when you resize a panel , move something etc...

Indeed we need to be sure that the call is not done to many time, and I think we need to review a little bit the display component for this.

Another thought could be to define/update the dropdown ui-hint to add a "Refresh button" along the dropdown to force the refresh of a specific dropdown ?

[jeanbaptistedalle]

The refresh button could be helpfull to refresh only one field in case the datas changed and I don't want to reload the whole page, but I'm not sure this will solve the root cause of the issue : as a user, I think it will be painful to load an activity, see there is only 1 or 2 select loaded and have to click on each failed select to refresh the selects with the correct list of options, especially if that happens every time I load the workflow for the first time in a browser tab.