September 2023 – Batching in Salesforce Flows

When I first started working on Flows, I read this post, which has remained incredibly useful even as Flows have changed and improved. I’ve come back to it several times in the last few months, as I worked on two Screen Flows that needed to do some expensive things in a loop.

I know what you’re thinking – no DML elements in loops! In general, I totally agree. I take seriously the argument that this should have been done with code instead of clicks to avoid this kind of thing. I also take seriously that a Flow is significantly easier to maintain and tweak than code, and that in other ways, a Screen Flow was a perfect tool for these problems.

In both cases, I was starting with external data that needed to be brought in to Salesforce. In both cases, the source data is not great and we have no power to change that. If we’re looking for great, we get rid of these data sources entirely. For a variety of reasons, that’s not an option, so what’s our next best scenario?

The next best scenario is a situation where we can programmatically update the data to be high-quality and ready to go into Salesforce, and then import it automatically. I love that idea. I’m involved in projects like that all the time.

For both of these problems, that option was off the table, for reasons I won’t get into here.

So what next? We could have someone manually update the data and then import it. That’s a real option and one we considered. But in both cases, I wound up believing that a Screen Flow to guide a user through how to update the data, and then having Salesforce update the data automatically was better. In both cases, some human review of the data and decisions made about it were important. Perhaps someday Salesforce’s product that is going to integrate everything easily will solve this problem! In the meantime, a Screen Flow is a pretty good solution.

The advantages of a Screen Flow were how easy they are to build and maintain and change over time, and how easy it is to let users interact with data. Giving up these two advantages would have made these projects prohibitively expensive over time.

The disadvantages were apparent from the start – with the limits in Flows, we wouldn’t be able to save hundreds of records at a time, which was important.

While I felt guilty about including DML elements in a loop for one these Flows, I wound up believing that this was the least bad of all our options. This Screen Flow takes a file that arrives periodically from an external payment processor and guides the user through finding matching donors and creating Opportunities and Payments. (UnofficialSF’s terrific Improved File Import tool was so helpful with this!) Unfortunately the donor information in the file is sometimes definitive and sometimes missing entirely, and so that review by the user was important. For each row in the file, the Flow would search for matching donors and then present the options to the user. The user could choose one of the matches or choose to create a new donor record. While this is tedious, it is still better than doing those searches and creating the Opportunties by hand. Once that review step was over, the Flow is left with collections of Opportunities and Payments that now need to be saved, and while this part wasn’t in a loop, that step turned out to be too expensive for the Flow with some files. To solve this, I batched the records to save and did it in small chunks. To force the Flow to complete the batches, I added a screen between batches.

The user didn’t have any need to interact with these screens – they existed purely to deal with the limits. Since the user had already stepped through lots of screens to choose donors, I didn’t want to ask them to do more work. Fortunately, UnofficialSF has a solution – the Auto Navigate component. This lets the user go off and get a cup of coffee while the Flow finishes up.

Each of these Flows makes me a little queasy – they’re not what I consider best practices and I look forward to the day when Flows are so sophisticated that I can update them to be something I’d really be proud of. What I know is that these flawed screen Flows are a big improvement over the manual data entry that was required previously, were possible to do on a very small budget, and can be changed by multiple members of the team. Sometimes pretty good is just fine.

I also took advantage of UnofficialSF’s . As usual, UnofficialSF’s Flow components are incredibly useful.

Leave a comment