Week 3001

Work

I managed to bring down a server at work while trying to do some experiments with multiprocessing. I also found that chunking out the data, starting a process and getting the data back all seemed to add up to a bigger time cost per chunk than the multiprocessing saved. This is using multiple workers to handle tax lot processing. When I can find time I’ll see if I can whittle it down more, but that was a very surprising note – if you don’t have a big chunk to process for each worker the coordination can be a bigger transaction cost. Just more times I see applications of the Ronald Coase’s work.

Caught up with some folks from  old workplaces and got lots of juicy gossip. What do you do when your workplace is being chaotic? Keeping your head down and getting things done always gets better results than conniving and rumor mongering. Still – chaos at work is distracting. Everyone wants to meet up at the coffee machine to talk about the latest development.

One thing that I’ve never really seen work well is Power Moves. Supposedly some people can do them, but I usually see them backfire. People don’t like being pushed, and they push back. Generally, people do like being led – it’s nice to know what direction we are moving in and to move together.

In general, I find that you can push and fight, but if you find yourself fighting, you have to question if you need to be in the fight. I do the best when I look at my employers as equals – I try to be a good bang for the buck they pay me. If I generally agree with the direction we are going I play my position, help out where I can and move us forward. If they don’t want what I’m selling or want me to do something I find repugnant – I go work with someone else. It isn’t worth it to fight when there’s such a big market out there. My job isn’t to make massive efforts to change the minds of the folks making decisions. They need to earn me as much as I need to earn them. Anything else and you aren’t in a good relationship.

I was also reminded of a good pattern for announcing changes at work. When you make any change, don’t send an email telling a group of people about the change. Instead, make it your practice to update the internal wiki docs and then send a link to the doc with the updates and maybe a summary. That way you don’t let the internal docs get out of date! People will rarely read your release email, but they’ll go to the docs when they are trying to get something done. Keep the docs up to date so they don’t give up and go to your support folks first. It’s a savings for everyone!

I’m also trying to get more of a culture around code review on my team. We’re all advanced tech folks and very good at what we do. But I think it’s always important to review together as a team. “Code should be written by a team and look like it was written by a single individual.” Code reviews are a tiny transaction cost (COASE AGAIN!) but they are a big review. I fully endorse my friend Nina’s talk on How to grow a Code Review culture and their benefits.


Life

I find I am missing the people I care about that live far away a lot this week. I think my zillionaire dream would be to put all my favorite people as close together as possible. I’d live with them and play with them. But I also know they need their people. So my dream ends up a dense metropolis of cool folks – an NYC of my tribe. But many people in my tribe don’t want to live in a metropolis. Portals would be helpful.

Took Max Lazer into the Take Your Kids to Work day this week and it was great. He didn’t always participate, but I think I coulda told them more about helping him with transitions. Otherwise he was really interested in building robots and playing at work with Dad.

Other

All of my work on performance improvements reminds me that we are just trying to get computing to resemble what we see in real life: instant feedback.

We do that naturally with physical things, and I like to see examples of physical computing. Why isn’t everything like this?

Physical computing is Super Different. Immediate feedback matters.

I guess this is one of the reasons I really like Jupyter Lab, and I’m writing a solution to my problems with DreamHost in a Jupyter Lab notebook. More details as soon as I have something. Which better be before October!

I installed the latest ubunto on a Lenovo Yoga 920 and it’s friggin awesome. Highly recommend, very easy. I can segregate work stuff fully from home stuff.

Also, on my reading list: A whole magazine issue dedicated to documentation in tech world.

I managed 49 miles this week on the bike! Commuted every day but Take Your Kid to Work day. I’m proud of it. Definitely more proud than I am of bringing down a server.

But wait, there's more

2 thoughts on “Week 3001

  1. What kind of parallel calculations were you doing? I recently re-built a code to go from openMP to MPI, and the speedup was huge. I was blown away by how much overhead goes into starting a loop in openMP.

    I’ve enjoyed the resurgence of old-style blogging!

  2. I was asked to improve some python that estimates realized pnl on a portfolio from some trades. Given a trade and some open tax lots each trade could close lots independently of other trades. Lots must be closed in order, first in first out style.

    It seemed like if I had 10000 open positions made up of around 100000 lots I actually spent more time pickling and unpickling to create new processes.

    If I were doing more perf on this I might just bunk it into x chunks and then spin up a pool of x processes to chew through them.

Leave a Reply

Your email address will not be published. Required fields are marked *