Computers, Music, and Thoughts
There are several guides from Salesforce and other blogs on how to run Salesforce Data Loader from the command line. In addition you can script sfdx to do data loads as well but it seems that only supports bulk which for certain objects can be buggy so we might explore that later for this entry…
CSVFIX has been my go to for command line ETL but I have always run it on Windows. After running in to some issues with a CSV file loading in to Salesforce I needed something on the mac to ‘peak’ in to the file and do the usual amount of stuff that I’m used to…
One tool I use to do bulk data loads in to Salesforce.com does not allow mapping one source to multiple destination fields. You can of course use workflows or code to backfill the data once it gets in but a cheaper/easier alternative if it is just one field is the following: // Works on Windows…
If you are using CSVFIX and running UNIQUE against a large data set you have to remember that CSVFIX loads the entire file in to memory to do the processing. So if you get this…check and watch your environments memory. Mine only had 3gig…it could use a little more for parsing 3+ million rows. ERROR:…
Sometimes I get SAP data that is quoted with both single quotes and double quotes. The data looks something like this: “‘001′”,”’01′”,”‘100123′”,”‘#’”,”‘#’”,”‘#’”,”‘100123′” CSVFIX will strip out the extra single quotes with the following command: csvfix read_dsv -s “,” -csv Input_File1.csv > Output_File1.csv Seems unneeded but otherwise your output is double quoted. I blame SAP. The…
Update Oct 2017 – For most csvfix commands the ‘-ifn’ command will also skip the first line. I say most as I know that it does not work for “read_dsv” which is ok as long as you are not combining files in addition of converting to CSV. The below solution is also good if you’re getting output…