17,500 Anagrams

Did you know that there are 17,500 anagrams for “George Hotelling”? If you are doing the math right now, you’ll probably come up with a different number because we are working off different word lists. For example, my word list includes “Google”. Did you know “Googler Eel Thing” is an anagram for “George Hotelling”?

Now that I have a list of anagrams, I wanted to automate changing my Twitter username to an anagram periodically. I set up a script to do that, but had to choose the frequency. After messing around for a bit, I realized that 17,500 was almost exactly the number of hours in 2 years (17,520 or 17,544 hours, depending on which years). So, for the next 2 years (or until Twitter blocks my script) (or until my bot fails) (or until I give up) you can pop over to @revgeorge for your hourly anagram fix.

Code is also up on GitHub.

“Rhino Egg Lot Glee” signing off.

Driverless Bus Prototype Coming to Ann Arbor

This is cool, a prototype autonomous bus is coming to Ann Arbor in the fall.

The service will use two fully automated, 15-passenger, all-electric shuttles manufactured by French firm NAVYA to transport students, faculty and staff on a nonstop two-mile route between the Lurie Engineering Center and the university’s North Campus Research Complex on Plymouth Road.

Mcity will study how passengers react to the vehicle as a way to gauge consumer acceptance of the technology. Exterior cameras will capture the reaction and behavior of other road users, especially bicyclists and pedestrians. Mcity will also track ridership and usage patterns, and survey users about their experience.The data gathered will help researchers understand how to design safer vehicles and how to operate them more efficiently.

The shuttle service will run on U-M roads during business hours to start. There will be no cost to riders, and the two shuttles will cover the route roughly every 10 minutes. Hours of operation and the service area could be increased later if the technology proves effective and consumer acceptance supports expansion.

The article says it’s a 2 mile route but this is my best guess for the route:

 

Tabs vs Spaces: Are we even arguing about the same thing?

The big news this week (other than all that political news) is objective data in the ongoing battle of Tabs vs Spaces. Stack Overflow’s annual developer survey uncovered an interesting correlation: collectively, devs who use spaces are earn more than those who use tabs.

salary_graph-1-1400x1000

That is sure to add some fire to never-ending war, but what if the data is bad? What if people don’t actually know which camp they’re in? Stay with me here.

To explain, first some back-story. Computers cannot actually deal with letters. Oh sure, it looks like you are reading letters on a computer, but they are actually numbers. Just like a kid’s A=1, B=2… cypher, computers represent letters as numbers. In the case of ASCII, A=65, B=66

So, when it came time to indent code, there were a couple obvious choices: space (32) or “horizontal tab” (9). 32 (space) was attractive because it was obvious, 9 (tab) was attractive because it was a single character that could be configured to be any width. The ultimate winner was anyone who could figure out how to turn this stupid nerd fight into pageviews – like me and Stack Overflow. Even Silicon Valley weighed in:

But what if… people didn’t actually know which one they use?

A lot of code editors recognize that people want to insert 32 four times when they press the “tab” key on their keyboard. These are frequently called “soft tabs”. With that setup, a person presses the “tab” key but inserts four space characters (32). Do these devs really think they are in the tab (9) camp?

When I first heard this theory from my friend Adam Kaump I was incredulous. But, it turns out there is evidence for this line of thinking:

Here’s someone who is saying that if you are pressing the “tab” key then you are in the “tab” (9) camp. Even if your editor is inserting four 32s.

Another someone else who apparently thinks that the argument is about which key is pressed, not 9 vs 32:

Doesnt using spaces to indent code waste a lot of time? I mean sure if you’re only indenting once, it takes a few extra key presses to make 1 tab worth of space bar clicks but if your code gets really deep, then you’re talking about wasting a lot of time hitting the space bar key per indent PER line…each subsequent line of indented code doubles the amount of space bar clicks…

Could it be that many who think they are in the Tab Camp are actually on Team Space? If anyone has any more examples of this line of thinking, please leave them in the comments.

As for me, I’m happy to be on the right side of this debate, and urge all devs to turn on their editor’s “visible whitespace” feature.

WordPress.com in Detroit

Automattic (my employer) has been partnering with Rebrand Detroit to get 100 local, small businesses online.

I’m thrilled that our international company is focusing on Detroit. If you are too (and you are reading this on May 17, 2017) you should come to the Rebrand Cities Salon this afternoon in Midtown.

Matt Mullenweg, the CEO, just posted these new ads showing some of the work that has been done so far:

Workaround for IKEA Bekant Sit/Stand Desks

When I set up my home office in 2015, I knew I wanted a motorized standing desk. The easier it is to raise and lower, the less commitment there is in standing up. The less commitment, the easier it is to stand up. At the time, IKEA had just released their Bekant Sit/Stand desk for about 2/3 the cost of the competition. I wish I had paid more to get a better desk.

The problem: It’s well documented that the Bekant has a power supply issue. The symptom is that it stops after raising just a few millimeters, then later refuses to move at all. This is caused by the power supply not putting out enough juice, according to some reports. My desk got stuck in a standing position for a week, maybe the longest work week I’ve had.

The workaround: Unplug the power supply when you are not actively raising or lowering the desk. Leave it unplugged until the moment you feel like switching between sitting and standing. I’ve been doing this for the past week and have had extremely consistent success. My power supply is hanging in the netting under the desk so I just need to pop the cord into that. No need to mess with the outlet.

The real fix: Keep returning your power supply to IKEA until you get one that works. Some people report never having an issue, and maybe you will get a power supply as glorious as theirs.

The good news is that the desk has a 10 year warranty – if you have your receipt. And your return will be much smoother if you get an employee who doesn’t demand that you disassemble your entire desk just for a detachable power supply.

I was lucky to have an employee tell me that they accept scanned copies of receipts, so mine lives in Evernote and comes out once a year when I want to try to fix my desk yet again. It takes about an hour to do the exchange in my experience, and I’m on my 3rd power supply. I also happen to live 10 minutes from an IKEA; I can’t imagine driving a couple of hours to do the exchange.

You may have some luck in contacting Rol Ergo directly for a replacement, according to this Facebook comment.

[Photo credit: mastermaq under CC 2.0 BY-SA license. Because my desk is in no condition be seen by anyone.]

Coworking at Workantile

One of the challenges of distributed work is isolation. Our third places are already disappearing, losing the office as a second place can be tough. So, about once a week, I drive over half an hour to Workantile, a coworking space in Ann Arbor.

It started with the Ann Arbor Software Co-Workers. I attended a few of those meetups at a coffee shop and was reminded that there is something energizing about being around people. At the same time, I wanted somewhere that I could hop on a video call. I wanted somewhere I didn’t have to make sure that someone watched my bag when I went to the bathroom. Somewhere I didn’t have to OD on caffeine to make sure I’m paying my rent. So I wound up at Workantile.

[As an aside, my employer has a coworking stipend that covers traditional coworking spaces as well as coffee rent at coffee shops. Companies that want to support distributed workers, take note.]

Workantile is part of a vanishing breed. There are plenty of coworking spaces that are “you pay money, get desk, end of story.” There are also plenty of incubators that want to host your startup. What Workantile focuses on are the independent contractors and the remote workers that want a second place. Here’s how they put it:

At Workantile our goal is to support the work and social needs of independent workers typically already established in their job or business, whether they’re self-employed or remote employees. Workantile creates a diverse workplace environment without the office politics. Our social network isn’t for smarmy business-oriented “networking,” but for real social interaction and camaraderie.

Quartz has a great write-up on how coworking has moved from the Workantile model to the other models. Companies like WeWork have been focused on flexible office space, ignoring any sort of community aspect. That probably makes financial sense, but I’m not really interested in “Office Hotelling.” It’s kind of disheartening that as more workers are becoming remote and distributed, social coworking is on its way out.

There’s a new coworking space “Pastel” opening much closer to me – a 5 minute commute instead of a 35 minute commute. Their vision is “a place for independent business women to find support, balance and connection.” That’s not quite me, but I’m glad to see other coworking spaces opening to focus on something other than real estate. I will be keeping an eye on them but for now it sounds like Workantile is a better fit.

Don’t get me wrong: I still loathe commutes but once a week is the right balance for me right now.

[Photo credit Chris Salzman]

My Net Neutrality Letter

[Below is the letter I sent to my representative, my senators, and the FCC. Well, I will send it to the FCC when their site stops being overloaded. It seems that John Oliver told the entire internet to Go FCC Yourself and they did. And then someone also spammed the site with purloined identities and anti-Net-Neutrality comments. So the FCC site is a little slow right now. But once it gets back on its feet, this baby is headed right there too.]

I want to voice my support for Net Neutrality. I am a software engineer in your district, and make my livelihood on the open internet.

I can appreciate the concerns of people opposed to Net Neutrality. I agree with them that efficient markets generally provide better outcomes than centralized control for most people. I can sympathize with the attraction of “zero rating” – letting people use some services without counting against their bandwidth. Ideally we should be able to continue to benefit from the phenomenal economic growth that the internet has provided.

Internet service is not an efficient market today. Until it is, we need Net Neutrality. I am fortunate to have 2 high-speed ISPs available to me, which makes me part of the only 10% of Americans with that choice. Markets don’t work when 90% of the consumers don’t have a choice. This Econ 101 principle is clear in an industry that is both highly profitable and consistently hated by their customers.

Net Neutrality also ensures that the internet can stay fertile ground for new businesses. If ISPs are allowed to provide incumbents with exclusive traffic, the cost to disrupt goes up. Would Google or Facebook have been able to grow if they had been crushed in their infancy by Alta Vista and MySpace? I want to ensure that the services I love 10 years from now are able to start today.

Net Neutrality is a key piece to the internet’s economic engine. Please keep the internet a Title II service.

[The John Oliver piece is below, I imagine the FCC is already aware of it so I chose not to include it.]

iOS Widgets for Home Assistant with Workflow

One of the key principles of home automation is being able to control things quickly and easily. I don’t want to have to unlock my phone, open an app, wait for it to load, and then finally find the device I want to control.

I like that SmartThings supports the Today screen widgets in iOS. They let me swipe dow on my locked phone and trigger a routine. For example, I have a button set up to run “Good Night” when I get to bed, which turns off lights, locks the door, and sets the thermostat.

I wanted the same in Home Assistant. It turns out that Workflow is perfect for this.

If you have the Workflow app installed, here’s an example workflow you can customize.

To make this, I created a new “Today Widget” workflow in the app. Basically we are going to create a call to the Home Assistant RESTful API, so to start you need to add a URL action. This should be in the form of https://your-home-assistant-site/api/services/domain/service. For example, if we want to toggle the bedroom lights of the Home Assistant demo site we would put in the URL https://home-assistant.io/demo/api/services/light/toggle

Next, add the “Get Contents of URL” action. Set the Method to POST, under headers add “x-ha-access” as the key and your Home Assistant password as the value. Set the Request Body to JSON and add any service parameters in the Request body. In our example we send a text value for “entity_id” as “group.bedroom”.

At this point you should be all ready to try it out! Here’s what it looks like:

IMG_2582.PNG

(note that is not my real passphrase)

Workflow is a great way to build easy-access buttons to run things in Home Assistant. Along with the Home Assistant native app, I think Workflow is a must-have for any iOS user.

Speaking at Self.conference 2017

I will be speaking about home automation with Home Assistant at Self.conference in May! One of my goals this year is to do more public speaking, so I’m thrilled to get to be part of this event. There is a pretty amazing lineup of speakers so you should probably go ahead and register already.

Here’s the write-up of my talk:

A Smart Home To Call Your Own

The stuff in your home is getting smarter every day. It’s time to take control of it with Home Assistant, a Python open source app.

This talk will provide an overview of how Home Assistant can be set up and configured to make your home smarter. You’ll be able to control your smart home devices so that your home starts working for you. With presence detection, logging, and mobile access, you’ll be the master of your domicile.

Once we see what Home Assistant can do we will explore how to write a custom components to start automating our own devices. Welcome to the world of tomorrow!

My Favorite Bug

Years ago, at a financial services company…

It started with some mild grumbling. Some people were complaining about getting signed out on our customer portal. But it was probably just user error, right? The reports seemed to happen more and more though. Then, employees who worked on the product started to get signed out.

At this point we have to admit that there is a problem. Every so often someone is using the site and they are suddenly booted out.

It happens seemingly at random. If you have a bug that you cannot reliably reproduce, your goal is to gather enough information until you can. “Seemingly at random” is a euphemism for “caused by some piece that we aren’t considering.”

Being able to reproduce a bug is important. First, you want to be confident that you have fixed the issue. If you can’t be sure you’ll trigger the bug, you can’t be sure your fix worked. Second, the factors that go into triggering the bug give you insight into where the problem is. Working toward a consistent reproduction is helping you triangulate the problem. So we started gathering information in hopes that we could reproduce it.

One thing you can do is ask yourself “what changed?” One of the things that changed was that we were rolling out a new version of our in-house PHP framework. This felt important, but also not, since the affected site was on the old framework. What’s more, both frameworks used the same way to delete sessions. Something like:

sprintf('DELETE FROM `sessions` WHERE `expires_at` < %d', now());

Which is a pretty reasonable way to expire sessions. The session creation code was similarly reasonable

sprintf('INSERT INTO `sessions` (`cookie`, …, `expires_at`)  VALUES (:cookie, …, :expires )', $cookie, …, now() + SESSION_LENGTH);

Jason, one of our analytics folks, was able to add HTTP monitoring. We figured out what the problem looked like from the browser side (a request with a session id setting a different session id in less than n seconds) and started logging how often it happened.

What we learned from that the frequency was accelerating. The longer it took us to fix, the worse it was going to get.

We still cannot reliably reproduce this at this point, so we have to keep gathering information. I was tempted to deploy changes out to see if they worked. This is dangerous. If you don’t understand the problem, you don’t know if a fix is going to inadvertently make things worse. Remember: if it seems random, you don’t understand it.

Adam, one of the devs on the team, added logging to our session delete code. Before it ran the DELETE statement, it did something like

debug_log(query(sprintf('SELECT cookie, expires_at FROM `sessions` WHERE `expires_at` < %d', now())));

so we could see exactly which sessions were being deleted. That’s when things started to get weird.

We would compare that to our HTTP monitoring and find the session and see that it definitely should not have expired based on the timing. But the session logging says that it definitely should be deleted based on the time it was logged.


At this point we reach the “Eureka” moment. For context, “Eureka” is what old Archimedes shouted as he jumped out of the bathtub. While the bathtub is clearly useful in realizing that water displacement can measure volume, I don’t think enough credit is given to the bathtub as a place of thought in that story. I consider showering (along with dog walking, driving, and sleeping) to be an important part of debugging.

And so I was, when I was trying to figure out how to reconcile the two facts:

  1. Sessions were expiring before they should be
  2. Sessions were expiring on time

I realized the disconnect, hopped out of the shower and sent an email to our Unix admins asking them to check the clocks on our production servers. Sure enough, one of the load-balanced servers wasn’t running ntp and it’s clock had drifted out of sync.

What was happening was if the session code was called on the rogue server then now() was wrong. That meant that sessions created on it would live extra long, but sessions deleted on it would die young.

Why was it accelerating? It turns out that in V1 of our framework the session-delete code was mistakenly not hooked up, but that was fixed in V2. The way PHP works is that every n requests it calls the cleanup function. We were moving more and more traffic to V2, which meant that the cleanup function was being called more often.


I like debugging. Well, I like having debugged. It can be maddening in the middle of it but there is a lot of satisfaction in solving a puzzle. A bug is a puzzle with stakes. If you are spending your time on a bug, that means its important. There’s also satisfaction in finally arriving at a simple, testable, theory of a bug.

I like this bug because it required us to understand so many pieces that make up the system: HTTP requests, session persistence, load balancing, and server clocks. It’s also nice when the fix is to adjust the clock instead of recode your entire app.

Image credit: Public Domain image NH 96566-KN The First “Computer Bug”