Tag Archives: Python

Backing up a SalesForce instance

SalesForce is an interesting beast. You gotta work within its limits, and it is great within them. As soon as you want to venture outside of the normal flow, things get complicated.

They suck  into SalesForce, but never out – it’s designed as a lobster trap for your information.

Weirdly, there’s not much on the SalesForce AppExchange that helps you easily back up your data on site. There are some tools that help you easily back up to another cloud, but little that helps you get your data back within your own walls.

Still, there’s a little layer over the SalesForce API in Python called simple-salesforce. Here’s a quick script I threw together to help put all your data into csv files.

I love PETL

When I started at my current job I noticed we we had lots of room for improvement about how we imported and exported data.  Folks had been using the MicroSoft SSIS platform as a way to Extract, Transform and Load data in and out of our database to various files.

SSIS is great for lots of things and has a lot of upsides. It is very drag and drop, folks don’t have to know a lot of programming to get it to do things, and it has lots of functions built in. If you need more programming power, you can execute C# or VB scripts to do the fiddly bits.

But I hate it. ( Don’t worry, we’ll get to the love soon.)

My biggest problems with SSIS:

  • It is unversionable. Try reading a git diff of an SSIS change. The xml is designed for a machine to read, not a human. If you want to know what has changed over time in your world, it’s a problem.
  • You can only use Visual Studio to edit it. Many of our SSIS packages include VB or C# scripts. That sounds fine – but apparently these compile to an undiffable, uneditable blob in the xml that is only recompiled if you save using visual studio. So if you want to change something across many SSIS package scripts, you have to open and resave each one.
  • It hides options under rocks. Finding out how something works requires lots of delving into lotsa windows and dialogues.
  • It changes things unexpectedly. Click in the wrong dialogue and it helpfully re-infers datatypes from a file for you. You don’t know until you go to execute.
  • It slapped my momma. Etc.

I wanted to move my team to something that was better for people.

We need something:

  • That we can diff
  • That we can do code reviews and pull requests on
  • That is simple, expressive and clear.
  • That is powerful.

To me that sounds like a programming language.  I encouraged folks on the team to try accomplishing a couple of tasks that might use an SSIS package instead to use Python. Immediately, things got better. Our code reviews made sense. Code quality improved with every single pull request.

We used pymssql to connect to SqlServer and inserted records as needed after processing them. Navigating and transforming XML docs was easy, CSV files were eaten up by the native DictReader.

And then Derrick found PETL. It’s beautiful. You point it at data and make simple moves to completely transform it. I’m smitten.

I had dozens of files to read from, each a quarterly file for a year – only noted in the file name. Each had a crappy heading line that preceded column headers. I needed to put them into 1 file for loading into SalesForce Wave. Whacking together a solution with PETL was effortless. Line 36 is where the PETL starts, and it’s so small and good that it is nice to see how much it encapsulates.

How to migrate your WordPress Blog between hosts.

My boss Mike needed to move his wine review blog from a friend’s hosting on lunarpages. I suggested he try dreamhost and he liked it – in a few minutes he had signed up for a free trial and used their 1-click install to set up a new install of wordpress.

Before he moved his domain to point from lunarpages to dreamhost I got him to prep by writing down a few important pieces of info. I’m trying to make sure I make this easier for other friends like I did when I helped Tove’s Thread For Thought move from WordPress.org to her own host.

Things to do before you change your domain to point to your new hosting

      Write down the name of your THEME. If you want to use the same theme, it’s important to write this down before you make the switch.
      Export your blog content from wordpress.
      Download your images. The wordpress export guide pretends this is easy, but it isn’t. If you are using the same domain name, I’m not sure what the easy way to do this is.

How to download your images

I wrote a python script that does this for you.
Make sure your system supports python. Next install BeautifulSoup – a great html parser for python.
Once that’s done, download this little script and change home and filesUrl to be your domain name.
Run the script, it should crawl your domain and download all of the images you host. Now follow the same steps of editing your export if needed and upload it all into your new blog at your new domain.

Hope that helps!

Tweetability and Readability

Robin wrote a good story: Last Beautiful.  http://ur1.ca/sees
I thought Robin’s writing was “twitterish” – most of the sentences were under 140 characters. http://ur1.ca/seex
Maybe microblogging pushed him to write better, snappier sentences? http://ur1.ca/sewq
Sounds like a hypothesis – let’s make that story easy to parse: http://ur1.ca/sefl
Time to break out the python and turn this story into tweets: http://gist.github.com/348616

Continue reading Tweetability and Readability

Mixins, Django, Python and adding to Models

I’m working with my buddy ADubs, esq. on a project where we make it easier to add federation to various models. We’re properly lazy programmers, so we don’t want to rewrite federation for each model we are creating, we just want to mix it in as a general set of functionality.  Below the fold, more details, sample code, etc.
Continue reading Mixins, Django, Python and adding to Models