Category Archives: Dev

Backing up a SalesForce instance

SalesForce is an interesting beast. You gotta work within its limits, and it is great within them. As soon as you want to venture outside of the normal flow, things get complicated.

They suck  into SalesForce, but never out – it’s designed as a lobster trap for your information.

Weirdly, there’s not much on the SalesForce AppExchange that helps you easily back up your data on site. There are some tools that help you easily back up to another cloud, but little that helps you get your data back within your own walls.

Still, there’s a little layer over the SalesForce API in Python called simple-salesforce. Here’s a quick script I threw together to help put all your data into csv files.

I love PETL

When I started at my current job I noticed we we had lots of room for improvement about how we imported and exported data.  Folks had been using the MicroSoft SSIS platform as a way to Extract, Transform and Load data in and out of our database to various files.

SSIS is great for lots of things and has a lot of upsides. It is very drag and drop, folks don’t have to know a lot of programming to get it to do things, and it has lots of functions built in. If you need more programming power, you can execute C# or VB scripts to do the fiddly bits.

But I hate it. ( Don’t worry, we’ll get to the love soon.)

My biggest problems with SSIS:

  • It is unversionable. Try reading a git diff of an SSIS change. The xml is designed for a machine to read, not a human. If you want to know what has changed over time in your world, it’s a problem.
  • You can only use Visual Studio to edit it. Many of our SSIS packages include VB or C# scripts. That sounds fine – but apparently these compile to an undiffable, uneditable blob in the xml that is only recompiled if you save using visual studio. So if you want to change something across many SSIS package scripts, you have to open and resave each one.
  • It hides options under rocks. Finding out how something works requires lots of delving into lotsa windows and dialogues.
  • It changes things unexpectedly. Click in the wrong dialogue and it helpfully re-infers datatypes from a file for you. You don’t know until you go to execute.
  • It slapped my momma. Etc.

I wanted to move my team to something that was better for people.

We need something:

  • That we can diff
  • That we can do code reviews and pull requests on
  • That is simple, expressive and clear.
  • That is powerful.

To me that sounds like a programming language.  I encouraged folks on the team to try accomplishing a couple of tasks that might use an SSIS package instead to use Python. Immediately, things got better. Our code reviews made sense. Code quality improved with every single pull request.

We used pymssql to connect to SqlServer and inserted records as needed after processing them. Navigating and transforming XML docs was easy, CSV files were eaten up by the native DictReader.

And then Derrick found PETL. It’s beautiful. You point it at data and make simple moves to completely transform it. I’m smitten.

I had dozens of files to read from, each a quarterly file for a year – only noted in the file name. Each had a crappy heading line that preceded column headers. I needed to put them into 1 file for loading into SalesForce Wave. Whacking together a solution with PETL was effortless. Line 36 is where the PETL starts, and it’s so small and good that it is nice to see how much it encapsulates.

Quick Project Names Demo

At work, I’m trying to convince people that we should auto-generate at least a suggested code name for our project names. It’s an important thing for compliance and secrecy. You’d rather someone is overheard talking in the elevator about “Project Icy Gneiss” than about “the restructuring of Acme Corp”.

I wanted to make the point that if you just have a small list of adjectives and nouns you quickly get a vast space of possible names – more than we’ll exhaust.  But a working demo is more persuasive than logic.

I knocked this together last night: Projects-a-Plenty.

projects-a-plenty

Used bootstrap & angular which is kind of overkill on something this tiny.

Mistakes were made, tears were shed

Orbital 0.2.2 – Mistakes Were Made

 

This is a tweak to fix some problems people told me about on the Orbital Feed Reader GitHub Issues page.

Some folks were getting a very bad experience during the install – everything goes blank – or they were able to install and then couldn’t add feeds or read feeds. Not good for a feed reader!  I traced down a problem where if you had a specific new default collation on your database, the way I initialized the database tables made everything go boom.  It’s fixed now, but it’s part of the problem of supporting a lot of different configurations without a giant automated test lab.

The other was that someone pointed out there is an improved PressThis page in the latest version of WordPress.  Everything was working, but you’d get a nudge message saying that there was a new way to use PressThis and why are you going this old way?  Updated and I think it’s a really good experience now!

I also did some cleanup and logging improvements so that I can work with folks who have problems in a better way.

From Jekyll to Hugo

Sometimes simpler is better.

After being away from Orbital for a bit, I started revving my engines a bit by trying a new project on the subway and hacknight. Project Munchausen is about doing some writing and exploring some ideas – so I tried to get myself away from doing too much programming on it. Since it is just writing, I wanted to avoid falling into the trap of using a big content management system like WordPress.

I started exploring with the first Static Site Generator I know of: Jekyll. Jekyll powers the Pages feature on GitHub. I got really far, but felt really constrained. It felt like Jekyll was trying to push me to turn my site into a blog, like the downhill path of Jekyll was to create a blog.

I didn’t want to create a blog – I want to make something that’s an exploration of a subject. It will change over time, so it should have an RSS feed generated, but I didn’t want to only have a timeline view of things.

I don’t know how I happened on Hugo but it seemed perfect. The templates are simple, Go is a pretty simple language to use, and the organization of the content is really easy to understand.

In Hugo, you have a directory named content. Under that directory, any other directories are Sections.  Content goes in a section as a Markdown file. Templates can iterate over those sections and their contents.  Easy peasy.

So far, I’ve gotten farther with Hugo in a few subway rides than I have with long sessions wrangling Jekyll. My goal is to put something up on the web in a month.

Orbital Feed Reader 0.1.9 Codenamed “BORN FREE” is out!

Last night I pushed out the final changes to uncage version 0.1.9 and get it out in the wild.
MAX LAZER tears off on his own  to explore Oregon

This version is mostly about giving you freedom. You should be able to easily import your stuff from other platforms into Orbital Feed Reader to use it. If you decide that you don’t want to use Orbital Feed Reader any more, you should have an easy way to leave. Any software that doesn’t tell you how to leave it is something you should be suspicious of.

Get Feeds IN

Share your OPML

For Feed Readers, the standard way to exchange lists of subscriptions is a standard called OPML. It’s an old format, but it’s how they all exchange lists of feeds. I’ve buffed up the OPML import to handle larger numbers of feeds. After you import the file you have a chance to look over the feeds and edit them, tag them or exclude them entirely. Hit save and they’ll all flow into the backend.

 

Get Feeds OUT

But we also don’t want Orbital to be a lobster trap for you. If you import that kind of data I also want you to be able to export that kind of data. Now Orbital is good at exporting your feeds. If you are logged in – we’ll export a file with all your public and private subscriptions and let you take that wherever you want. Some folks want to share their OPML publicly – if you aren’t logged in, hitting the same URL gives you only the public feeds. All your private stuff should stay safe!

Read Lotsa Feeds

Now that Orbital can handle tons of feeds coming in I wanted it to be easy to actually sort through them. Feed Tags now start off collapsed and you can expand them anytime by using the little twiddlers next to each. That’s a technical term. The twiddlers make it much more manageable to see 200 feeds over many subject areas.

Orbital Feed Reader 0.1.8 Codenamed “BEAUTY TREATMENT” is out!

IMG_1185

The most common feedback I’ve heard from folks on Orbital is that it’s not very good looking.  It does the job, but it isn’t a pleasure to look at.

I assume these people are all supermodels, so it looks like Orbital is doing great among the high fashion set.

I got out my scissors, called in the wardrobe department, applied polish and buffed with a chamois. This is the new smooth look, designed to blend in with the new admin themes of WordPress.

Hey, good lookin’!

 

Beauty comes in all thicknesses

Along the way, I’ve added some features – now it is responsive, cleaner and is even usable on my Galaxy S5. Check out this hot resizing action!

Future attractions and graceful aging

Coming soon are some cool things. First, I’m working on version 0.1.9, code named Born Free. It will deal with the how to leave Orbital and how to get into Orbital. I’d never want to trap you!

Second, WordPress is going to upgrade to version 4.0 soon. I’ll be getting Orbital ready for that, and the new REST API should make it easy to build a better mobile experience for Orbital.

Press for Orbital

Two new reviews for Orbital bring a smile to my face!
WordPress Plugins A-Z gave Orbital a 4/5 rating:

This is a great feed reader especially if you are doing research for a post or if you are like many people out there who write additional articles based on what they read. This make it easy to get a pull quote from an article and drop it right into a post. While the layout and functionality is a little raw in its appearance it seems to work very well and to give it a good go I added it to my secret blog site as a place to test some ranting on article and stories. And it seems to work quite well..

That’s great feedback and I feel like I do need to work on the style and appearance – it’s tough and I need help.

Jeff Chandler over at WPTavern also reviewed Orbital, comparing it to Feedly and Google Reader. It’s a lukewarm review – he doesn’t like the interface and feels like Feedly plus PressThis is find for him. Still, he’s offered to help me by looking at revamps of the interface.