My update in January was about things that were cropping up around Enhanced Recurring Donations, and while I continue to find bits and bobs that are a bit annoying, I don’t have anything new and interesting to report on that front. I have been spending a lot of time deep in Recurring Donations, however, but at a different organization.
Back before NPSP existed, it wasn’t entirely obvious how to represent recurring donations in Salesforce! People tried some things. I’m currently working on overhauling one of those experiments that hasn’t aged well. I don’t think anyone involved in the design was incompetent, I just think it was a really difficult thing to understand all the consequences of since nobody had lived with any solution for any length of time!
What that means, however, is that we’re taking data that is radically different from Recurring Donations and child Opportunities and transforming it so that the organization can use NPSP Recurring Donations. We’ve now practiced twice exporting the data, deleting it from the sandbox, transforming it using pandas and then importing it back into Salesforce. I’ve never done such a radical transformation (this is way bigger than migrating from Raiser’s Edge or something like that,) and it has been very interesting.
One big lesson is that the Bulk API option in Data Loader makes something like this possible. I’m importing over 3 million new records, and we’re trying to keep our instance’s downtime to a few hours. We should be able to do that because of that Bulk API feature.
Another big lesson for me personally is that pandas is really well suited to this kind of work. I picked up pandas based on one (very wise) person’s somewhat offhand suggestion years ago, and I keep telling myself I should learn some other tools, but somehow the work I’m doing always seems to be something pandas is quite good for. Being able to do these pretty elegant transformations of 3 million rows on my laptop is great! (I’ll admit it is a very nice laptop.)
Artisanal Data 