On our tenth episode, we talk with with Mitchell Hashimoto, creator of Vagrant and Founder of HashiCorp, the force behind projects such as Packer, Serf, Consul and Terraform. Mitchell discusses the importance of not naming your company after your flagship product, why a good story is important for raising capital and how to handle hiring in your business’ early life. He also covers why it’s important to avoid moving too slow (even if it’s for the sake of sustainability), how to encourage word of mouth, his love of Go and much more. Listen to the podcast in the player above, or click here to download it directly. Subscribe on iTunes or do so manually by using this RSS feed.
Show notes
- Mitchell Hashimoto: Twitter, Github, Website
- HashiCorp: Website, Twitter, Github
- Vagrant
- Packer
- Consul
- Serf
- Terraform
Transcript
Alexis: Well, thank you Mitchell for taking time out of your schedule to join us in the show.
Mitchell: No problem. Happy to be here.
Alexis: Alright. For folks who aren’t familiar with you, give us a little bit of a lowdown of who you are and what you do.
Mitchell: So I guess I’m probably most well-known as the person who created some software called Vagrant. I’m also the founder of a company that started around that project called HashiCorp. And since then we built a series of kind of DevOps tools or tools on the DevOps category. They’re Packer, Serf, Consul and Terraform.
I also speak at a lot of conferences, so maybe someone has seen me there. And I’ve written a book for O’Reilly.
Alexis: Alright, now before we dive into how Vagrant came about, tell us a little bit about your background.
Mitchell: My background’s primarily, I mean it’s pretty industry heavy, it’s pretty standard. I graduated college in computer science, professionally I’ve just been in the industry for maybe seven or eight years. Not too long but my background is very much kind of startup-focused, and very automation-focused, so in my free time ever since I started programming, which is when I was 12, I’ve just been writing things to automate other things.
So when I was 12 it was me automating playing online games pretending to be human playing online games. And that turned into automating, registering for classes in college and university, which turned into automating finding the lowest book prices, which turned into Vagrant. And it’s just kind of gone from there.
Alexis: Alright. So out of curiosity, what games were you impersonating people on?
Mitchell: The biggest one, well the one that I guess my most infamous association with is Neopets. So I got a couple of cease and desist letters from Neopets.
Alexis: Yes. Hoping some World of Warcraft is in there.
Mitchell: Okay, so yes, I did that my Freshman year in college. It’s not a part of my personal history I usually talk about very much because I played a lot of World of Warcraft. I think, I would say at least — well they might have been private ones — but it’s the first World of Warcraft one for Mac that I know of.
Alexis: You’re saying.
Mitchell: Yes, I had this realization my freshmen year which is like, “Oh, World of Warcraft had all these anti-cheating mechanism on Windows, but on Mac they’re pretty limited.” Because they actually don’t run as root and I couldn’t find any process that ran as root in the World of Warcraft binary, so I was like like, “Well if they don’t run these root and the cheats runs as root, then it’s theoretically impossible for them to detect unless, you know, they catch you in the act. I wrote some software that ran an as roots and cheated World of Warcraft and it worked very well.
Alexis: Now, did you release that as open-source or did you just keep that.
Mitchell: No, I just kind of kept it. Right when I was working on that there was this popular World of Warcraft bot for Windows, like really popular, made millions of dollars called, I think it was called Wow Glider.
Alexis: Yes, that’s it.
Mitchell: Yes, and they were just in the process of being sued by Blizzard and I was not having any of that. So I didn’t release it.
Alexis: Alright, so when does, when does Vagrant come in the picture? Actually when does programming come into the picture just going, you know, I guess further back than college?
Mitchell: When I was twelve I actually Googled how to make an EXE. Everyone used Windows back then, nobody liked Mac then. I Googled that because I wanted to cheat at games. I had all these cheats that were EXEs that you double click — game trainers and stuff — and I had this realization one day which is that, “Hey someone makes these, like these don’t just exist. Someone has to make these. How does that work?” I just wanted to know how it worked. That kind of led me down the rabbit hole.
Alexis: Alright, so you’ve gone through the rabbit hole. You cheated World of War Craft , well, maybe not cheating, just –
Mitchell: I cheated.
Alexis: When does Vagrant come into the picture?
Mitchell: Vagrant comes in about my third year into university. For the three years prior to that I had been working for a Ruby on Rails Consultancy. I was just a developer there. Being part of a consultancy I saw a lot of different clients on a fairly routine basis and so I was getting frustrated with the fact that I had to basically reimage my machine for every client because the technologies were just incompatible at the same time.
So I was getting frustrated with that, and trying to figure out a way trying to run multiple ones. I tried various manual techniques basically on compiling with different paths and having config files at different ports and all that sort of stuff. That got tiring really fast so I was trying to figure out a way around that and Vagrant was my attempt at it, and it seemed to have stuck.
Alexis: So did you develop this personally or as part of the company?
Mitchell: No. I developed this just in my free time outside of the company.
Alexis: Was it an immediate decision, you never had to question that it was going to be open source or did that take some deliberation?
Mitchell: No. It was immediately open-sourced. I mean, I never thought it would be that big. I just wanted it for myself. I was pretty into the Ruby community – not known in the Ruby community — just into it. And I wrote it in Ruby and I was going to make it an open-source, because that was a very Ruby-thing to do. It was never a question.
Alexis: So how did it start to get traction?
Mitchell: Luck maybe. It didn’t have traction for a very long time. Well, a very long time being maybe like six months to a year. I remember for the first six months it probably maybe had a hundred downloads per version, which is abysmal.
I was excited when I reached a hundred downloads, and then I realized it wasn’t going higher than that, so then I kind of got disheartened a little bit. But, I think what really changed it is that I got a sponsorship from Engine Yard to speak at local conferences about it. And so I used that to kind of go to a handful of local conferences in the United States and talk about it and I think that was what started the process. It wasn’t like an overnight success or anything. I saw the downloads jump from a hundred, to two hundred, three hundred, five hundred, and then they kind of stuck around six hundred for a while but I mean that point is still a lot bigger, and then from there it just kind of slowly, slowly climbs in slope.
Alexis: It was more of a grass roots thing that started with the small kind of non-scalable way of marketing it going from one place to another talking about it. Then it just spread by word of mouth?
Mitchell: Yes. It was pretty grass roots. It was definitely not a, “build it and it will come” sort of thing. I don’t actually really believe in that philosophy. I think you have to market whatever you do so I think it was just conference speaking.
Alexis: So at what point did Vagrant start to take up more time than your day job?
Mitchell: Oh yes. There was an interesting point in Vagrant’s history, where probably into year two, I was spending probably an equal amount of time between my day job and Vagrant to the point where actually I got a little burnt-out. I actually took a few months — this was before Vagrant was very popular, it was just growing in popularity — and I actually barely worked on Vagrant except merging pull requests and fixing basic bugs. But I didn’t work on any feature for maybe three or four months and it was kind of an uncertain time, because Vagrant wasn’t big enough then.
I mean I loved it, but it did everything I wanted it to do. So it wasn’t big enough then that I felt like I really had an obligation to continue it and I wasn’t quite sure if I had the energy really to work on it as much as my job anymore.
It was kind of an uncertain time. One night I just got super into it again and started working on it and then it was maybe a year after — this is into to year two of Vagrant’s existence — and then into year three, is really when I decided that it was too much and I needed to do it full time if I wanted to. By then it was clearly what I loved to do, so I did it.
Alexis: Speaking of it gaining popularity nowadays, the BBC uses it, Disqus, Expedia, Mozilla, Yammer, Nokia, O’Reilly, and although you can’t probably officially mention it I have sources, previous interviews, that say that Disney may or may not be using it as well.
Mitchell: Maybe. I was invited to a Disney conference, so maybe.
Alexis: That may not be a very good tell. So you started HashiCorp. What was the, what went into the decision of making that the route that you follow to focus on Vagrant full time rather than some of the other options to support an open-source project?
Mitchell: Yeah, that was interesting. I struggled with that and by the time I decided to start working on this fulltime, I had given some hints out to some people I knew that I was heading down that path. I got a couple of job offers to just work on it fulltime, which was pretty flattering. I mean, that was my dream in college.
I actually have… I still saved it because it was so cool when I found it like year ago when I moved. But I actually have this piece of paper that I wrote when I was applying to college. And one of the questions on the application for the college computer science program was, “With your degree in Computer Science, what do you hope to accomplish?” My answer was, “I hope to be able to work on open source full time so that so people could use my software, and it’s free.”
And so, I kind of looked at it and I was like, “Cool! I’m getting the opportunity to achieve this goal.” So, I struggled with it but the main driving factor was that Vagrant, I felt was important. But I really wanted to try to solve more problems, and more problems were more things in the ops space. I felt that these people that were offering me a job wanted me to just work on Vagrant because they used it and they didn’t have any interest in me working on anything else.
And so I felt that having my own company was kind of the best way to try it. Because in the worst case scenario, we’re kind of privileged here as engineers. The worst case scenario is that, you know, is that it fails and I get another job. It’s not a do or die sort of thing, so I decided I was twenty-three or twenty-two at that time. I was like, “I’m just going to go for it, because I have nothing to lose.”
Alexis: Alright. So you left your job, you’re starting HashiCorp, what’s the monetization strategy that you are thinking of to dive into?
Mitchell: Yes. That was the major thing, the paychecks stopped coming, what to do? I had sold the prior business, so I felt pretty confident that I’d be able to cruise for maybe a year or two. I had that but I did want to start making money as soon as possible in some ways. So the first thing I did was create the VMware integration of Vagrant. You know a lot of people get confused about this, because they think that now the business model is paid plug-ins or something.
It really was less of a business model and more of an immediate need for money. It was something that I knew people would pay for and VMware cost money, so the integration costing money wasn’t really a slimy thing or, I don’t see it as a slimy thing because of that. And so I thought well, “Let me just do this because people will pay for it.” So, that’s what I did.
That actually grew into being able to pay multiple salaries so I was able to hire some people. I’m getting paid again which is a cool thing. Going past that you know that’s not the real business strategy, I think. You’ll see.
Alexis: It’s still unknown territory.
Mitchell: It’s kind of known, but I don’t want to prematurely do it before we have the actual thing we’re going to sell.
Alexis: Okay. I have to reposition some of my questions here to sail around that. So did you seek out any VC money or was this purely bootstrapped?
Mitchell: It was bootstrapped for a year. And then we did raise a small seed round. A fun fact is that we actually never spent any of that money. So I raised the small seed round because I was nervous, because if the VMware — my biggest existential fear at that time was that someone would make an open-source VMware plug-in and then all the sales of mine would drop off.
In the years since I’ve learned that that’s not a real concern because even if that would happen, I’ve learned that people actually want to pay you because they do want real support and they do want the comfort of knowing that it was made by the official people and things like that. So, it’s not a real concern. But at that time, it was a real concern.
And so I raised money so that I could pay other salaries in the case of a disaster but since then I learned it wasn’t a problem and the sales of the VMware thing actuall grew quite a bit. So, good things all around.
Alexis: So what kind of knowledge would you like to impart to folks who are about to embark on the search for VC money whether for an open-source project or a project of their own doing that’s not open-?
Mitchell: Yes, I have a lot of experience with this now now actually. The biggest thing I can say is that for a Seed Round or A Round, adoption could be used as an excuse for everything. I you have something with traction, you will have zero problems getting seed level money, like zero problems.
If it’s not a huge idea, if it’s not a super, secret idea, or IP-heavy idea, just try to get some traction to start. Show some growth and then it will be easy. Otherwise, you know, just have a good story, I think.
People want at the seed round especially, they want a good story because — especially at the seed round — it’s not about building a business. It’s more about building the idea out in Silicon Valley sort of view points. And so, they just want a good story and they want to know that you’re heading in the right direction.
Alexis: Okay so, a lot more marketing than anything. Yes.
Mitchell: Yes. I don’t’ know if this is true, because I never had to deal with this per se, but I had someone giving me advice, which is that if you actually had revenue for the Seed or even the A Round, it’s actually a really terrible thing because when there’s revenue… when there’s points A and B, then people will extrapolate Point C and see where you’re going. But if there’re no points on the table it’s just up to your imagination.
So if no one sees your revenue is, the expectations are much higher and you’ll actually probably get a better deal in terms of how much equity you’re giving away in your company and how much money you getting and things like that.
I didn’t have to deal with that so much because I was very up front with any VCs I talk to that any money we got from the VMware thing is just incidental and it’s not that real business model so it’s not a good measure. And that seem to have worked.
Alexis: You’ve mentioned retiring a couple of times. A lot of open-source projects go towards contributors who have been very dedicated when they weren’t being paid. Is that something you did as well?
Mitchell: So that’s something we’re doing right now. We’ve actually, at the this time of the talk, we’ve hired two others which haven’t started yet and there’s two more coming, that are they haven’t signed an offer yet, but I think they’ll be coming. That’s what we’re doing now. But no, when I first started the first hire I guess was actually my co-founder Armon. We went to college together and so I kind of hired from friends first. And Armon’s my best friend and he has a very complimentary skills set to mine, which is very academia focused, and a very deep computer science and so that’s been very helpful. He joined me early. And then our first hire after that was someone we worked with at the first company we worked together, Jack.
And so I think actually the best hires early on are actually people you’ve worked with before, because I think working in a for-pay environment is very different than working for fun. So I think it’s safer probably, at least initially, to hire people you’ve worked with before.
Alexis: Were those early hires developer focused or are they more business focused?
Mitchell: Yes, they were primarily developer focused. So Jack actually did a lot of business stuff, marketing sales. He started at the last company he worked at when he was 18 as a secretary, basically an assistant, so he’s kind of done full gamut of climbing the ladder from that to being basically a VP level type of person at a 30-50 person company. He kind of ran that whole thing, so he had a lot of experience but he’s also a very good developer.
Early on our focus was primarily on chipping products. He was very much developer focused, and still is to a certain extent, but he’s kind of turning into a product manager, more than anything, but he’s great at it.
Alexis: Now that you’re going more outside of the friends circle, what are the qualities that you look for when you’re hiring. Not just how much one has contributed to Vagrant or Packer, that kind of thing.
Mitchell: Yes, it’s tricky. I could probably safely say this because there’s around like twenty different contributors to our projects, but there are some contributors that just aren’t, I mean they’re great, they do great work but they aren’t really good candidates for hire, seemingly on the outside. What we look for most of all is actually passion for the space and that’s a tricky thing I guess to quantify, it’s more qualitative. Ops and DevOps is not a sexy space. I think it’s more so now than it was ever before. But I mean, it’s still not.
Finding someone who actually who blogs about and cares about it, has projects about or has worked only in that space, you know, is a really good quality to have. Because we’re all over the space and as a startup we’re going to be jumping all over, and we need to know that no matter where we jump to, that person doesn’t just care about working on a Ruby on Rails app, or doesn’t just care about working only on a monitoring thing. They have to be very flexible and care about helping the space as a whole.
Past the passion I think it’s just flexibility or malleability. We worked with Go and Ruby, and we work on development environments, production service discovery and infrastructure creation – it’s a lot of stuff there. And you just need to be able to adapt. We are not at the size where we can hire the person who knows Python really well and only works on this one Python thing for their entire career. Like we’re not at that size, so we need someone who’s flexible. And I think that’s really it. Those are the two main things.
Alexis: Sorry, Guido van Rossum, you heard it here first. You probably won’t be working for HashiCorp.
Mitchell: He knows C pretty well, so that’s pretty useful.
Alexis: Okay, let’s see.
Mitchell: Actually, fun fact in Python, my co-founder Armon, for a research project in college he actually wrote… he’s actually the author of Python interpreter or VM. I don’t know if it’s a VM or interpreter actually, the technology behind it. But he actually wrote one, he knows Python better than anyone I’ve ever met in my life. It’s really crazy. He embeds for all sorts of projects. And I didn’t even realize Python was easily embeddable. Oh, he says it’s not easily embeddable.
Alexis: He’s just that good.
Mitchell: He just does that. It’s actually kind of crazy. I mean he’s very good with Go now, extremely good with Go but he’s still busts out Python and does like prototypes extremely fast.
Alexis: Speaking of Go. So, Vagrant: Ruby. Now, Packer: Go. Serf: Go. Consul: Go. Terraform: Go. I’m seeing a pattern here.
Mitchell: Yes.
Alexis: What was behind this?
Mitchell: A lot of people ask that. Packer was released before Go really took off. I’d say it was probably in parallel with some other bigger Go projects but we kind of hit upon it at the same time. I didn’t so much choose Go as narrowed it down to being stuck with Go because I knew I didn’t want to do another big project in Ruby.
Ruby is tricky to distribute safely. There’s a lot of different versions. There’s a lot of dependency problems, there’s a lot of like C-extension and compile tool chains stuff required. I didn’t want to deal with that. I learned from from Vagrant and Ruby that people who aren’t familiar with Ruby community, which is most people, get really uncomfortable when there’s a gem install or something like that.
For a Ruby person it’s like, “Gem installs are so easy, why don’t you just do that?” It’s true for a Ruby person. But for everyone else, I got a lot of feedback early on, speaking in conferences in Vagrant that people just hate it whenever there’s a gem install, so I knew I didn’t want to go that route.
So I started narrowing it down and I didn’t want a JVM because I’m not familiar with the JVM. And also I think it’s kind of weird for a command line app. And then Python and Node kind of fell on the same category as Ruby, so I kind of want to avoid that. And really what I was left with realistically was C and kind of more obscure languages like Haskell or OCaml or something. I actually really like Haskell but I don’t claim to be able to be practical in it — some people can, but I can’t. So, that was kind off the table. It was kind of C and Rust that were left. Rust, still to this day is pretty experimental, and especially two years ago it was really experimental so that kind of self-eliminated itself.
And I was actually ready to write this thing in C. I’ve written C programs before and I was fine with that. I’d actually written C programs that had to compile to Windows. Linux and Mac, so I was fine with that. But I wasn’t looking forward to building the compiled tool chain like the auto tool stuff again because I kind of cargo cult that and I don’t understand it. And then Go was with there and I took a look at it and thought it might work. It was definitely was missing a lot of libraries.
I had a list of things that I would have to write on my own and it was pretty long. I was looking forward to that, but I decided to just give it a try because I felt that with the expressiveness of Go, that I could actually probably build all those libraries in the same amount of time that I could get something done with C. I don’t know if that’s true or not. But I tried it, and it kind of worked out. And since then I actually really, really like Go.
Alexis: Before we dive deeper, can you give us a 20,000 ft. view of the Packer, Serf, Consul and Terraform?
Mitchell: Yes. Sure. So, Packer is a tool that takes a configuration file and uses that file to create images or containers. So AMI’s, virtual machine images, Docker containers, Vagrant boxes. Stuff like that. It just kind of homogenizes, in a way, all these single platforms in a single tool to build images for it.
Serf is kind of strange. It’s a tool and a library for cluster management so, basically it notifies you when someone has joined the cluster, someone has left the cluster or someone has failed in the cluster and they’re not reachable anymore. And it does this in extremely fault-tolerant and scalable ways.
So Serf clusters scale up to many, many thousands of nodes that have been deployed and they won’t even show up on your network bandwidth, you won’t see them on the CPU graphs, it’s very scalable, very quiet.
And then Consul was actually built on top of Serf, so it uses Serf as a library rather than a application and is a full-service discovery, configuration and — to a limited extent — orchestration tool for your data center. Serf, just because it was so fault tolerant and stuff, it made those properties difficult.
So Consul adds a little bit of consistency on top of that. So you need Consul servers. Serf is completely decentralized, Consul has central servers and you use those. But again, Consul is really hard to bring down. That’s one of its leading traits. It’s really hard to lose data, or not get your data committed and it’s also really hard to take it down. And that’s also deployed to pretty large clusters. And then, Terraform.
Alexis: Which was announced out in July, right?
Mitchell: Yes. Like two weeks ago, three weeks ago maybe. Oh no, that was a month ago now. Terraform is two things. One thing, is it’s cloud formation kind of for anything, so for those who are not familiar with cloud formation, what Terraform wants to do is describe your infrastructure components in text, so, “I need a server, I need a load balance, I need an IP, I need this server’s IP to equal the elastic IP,” stuff like that. And Terraform actually builds the infrastructure for you. It doesn’t only do that but it also changes it for you.
So as you change the infrastructure it figures out what’s the difference and applies only the changes. So you could just kind if keep that in sync. Something new that Terraform does that cloud formation and other tools I haven’t seen really do, is it is able to tie together resources from multiple providers. So you could say, I want an EC2 instance from AWS, but I want a DNS entry in CloudFlare.
Alexis: Interesting.
Mitchell: And point to that instance and then I want a database from Heroku Postgres and I want that address to be used as the user data to configure that instance. You could tie these things together in a way that I think you haven’t been able to do before. And what some people have said publicly, it’s kind of like Terraform lets you not have to ever log in to these dashboards for all these software as a service ever again.
Alexis: Right. Which you have to deal with 20 different ones of those of varying quality of UI can be a pain.
Mitchell: Yes. I think, Terraform has probably had the most hype or excitement surrounding it than any of our products at launch time. And I think it’s because it really struck a nerve with that. We’re already getting a lot of inbound interest from software as a service providers that are asking if they could commission us basically to write providers for them so that they could advertise that sort of thing. Which I think is the way to go because I think we’ve reached this point where infrastructure as a service is being commoditized like Azure, Compute Engine, AWS, they’re all kind of converging at a core level to the same thing.
And so, they are trying to differentiate themselves based higher level services like databases and caches, load balancers and things like that. But as those higher level things come out you basically want to mix and match them and it’s really tricky to do that right now with automation.
Alexis: Just to clarify. In the past, I’ve read you would like to avoid services if you can and focus more on the products that you have. Now that you’ve got Packer, Serf, Consul, Terraform, and Vagrant, is HashiCorp at the place where it is more than breaking even or is that something that you’re waiting to hit when you unleash those kinds of new monetization strategies that you’ve mentioned briefly.
Mitchell: Yes. So we’re still avoiding services but to some extent services are a necessary evil at a certain point. I’ve always said we’ve avoided services but we don’t have a hard “no” on them. So we do some consulting and do some custom work and stuff for some people, we’re just very selective. It’s usually customers who are very big companies or we negotiate case studies out of them, or there has to be some additional kind of incentive to it. Going forward as we do get into more of a business snf model monetization strategy I think that support, services, integration consulting and things like that you just you have to have kind of as a stepping stone to sell your product.
So I think we will build out that team but it’s going to probably be, at least initially, with a selling component to it. For example, we’ll do integration consulting for you but only if you purchase one of our enterprise products.
Alexis: Right. So Vagrant has been around since 2010, now it’s 2014. Four long years — or maybe short years from your perspective — have passed. How has the community changed over time? And how has your interaction with it has changed as well?
Mitchell: The community has changed a bit. There are still the same core people but the earlier adopters I’ve noticed on any projects are always very technical, they’re willing to get dirty and solve their own problems, they’re willing to submit pull requests, things like that. As time has gone on, we’ve kind of reached this point where the masses have started adopting Vagrant. Trying to be as polite as possible, but the bar lowers and you get people who get error messages that describe exactly how to fix it and they still submit a bug. And it’s like the error message told you exactly what to do. So I don’t know what you want.
You get a lot more of that. And you get a lot more people who just simply don’t report bugs, they just work around it or they just don’t use our project anymore. And so that’s really tricky and a problem we’re still dealing with. How do you get those people to actually report a problem or how do you detect a problem and report it yourself?
But yes, I think as more people have used it, that’s happened and also the bugs we do get are really crazy because Vagrant and its core is quite stable now, reasonably stable. So the bugs we’re getting now are always — I tweeted about it maybe a or two ago — in the format of, “Vagrant on this host operating system, running this guest operating system, on this weird obscure hypervisor with a Chef provisioner with this file system layout results in this error.”
Alexis: And only on Tuesdays.
Mitchell: Yes. The bugs are actually really annoying. They’re obviously real bugs, but you know they’re going to take like an hour or two to really reproduce. And so it’s hard to justify giving up an hour or two of your time just to fix this problem, the problem that only a handful of people in the world have? But you have to fix them eventually so it’s… it’s weird.
Alexis: Now part of me would like to invent this fiction that Terraform actually sprung out of that need to debug those different environments.
Mitchell: You wouldn’t be far off the mark. Terraform was born before but the idea for Terraform has existed for many years between me and Armon. But, yes for years, Vagrant has an acceptance test framework. The problem is they’re so slow to run that we rarely run them and they also require physical hardware so we can’t run them on traditional clouds per se.
Alexis: Right.
Mitchell: But nowadays dedicated servers are getting API’s, so that’s interesting, but we always wished we had something like Terraform that could just spin up a dedicated server on demand and handle how to clean it up and stuff just so we could run this test. I really just need like this specific environment and I can’t run it in a virtual machine because Vagrant doesn’t run well on virtual machines because of nested virtual.
Alexis: Right.
Mitchell: Yes it’s stuff like that. We’re definitely actually putting Terraform to use in this sort of scenario soon.
Alexis: Alright. If you could distill those four years of experience working the HashiCorp community, what kind of advice would you give to folks who are working with communities on open-source projects?
Mitchell: The advice I give is early on, if you believe in it, stick with it. Listen to what people have to say because they are using probably just as much as you are so their experience is really important. There’s always going to be the mean people or the angry people, and the best advice I can give you there is write your response to the angry people and then don’t submit for at least two hours and look at it, reread it and then submit it. Because you might want to respond to anger with anger, but in the long run kindness is really going to be difficult to top. I think that’s kind of it. Just kind of do what you want.
Alexis: So after those four years how big has HashiCorp grown?
Mitchell: Oh, we’re still my three people. We have four more people coming on board like I said but at the moment we are just three people but we haven’t grown so much in employees as we have in usage. Year over year in terms of down load numbers Vagrant has been growing in the hundreds of percent. Many, many multipliers there.
Alexis: Now a question from the mailbag, actually from Reddit user duncanlock. He’s noticed how busy you were, so let me just read his comment/question. He says, “Mitchell H. is one of, if not the best, open-source project maintainers that I know of. Clearly insanely busy, yet unfailingly polite, helpful and encouraging to contributors.” And he adds, “Speaking of which, just look at his GitHub history. Just look at it. How is he so productive?” What he’s referencing is that you see a sea of green on your contributions and he adds, “How is he so productive? Look how many projects and companies he manages? How does he juggle all that? How many other people are involved? And do you have any tips, for being more like that?”
Mitchell: Yes. I would recommend not being more like that. I mean it’s what I love to do, so I do it a lot. I do have a life and I have a girlfriend and I have to juggle things like that. But my girlfriend knows she’s first but the computer is a pretty second close second and I spend a lot of time on it. So my normal working day, I’m a morning person so I get up at 7:30 and I shower, eat and I’m on the computer by 8 and then I’d go to the gym and things like that. I’m probably on the computer again until ten at night. I mean, that’s like a 14-hour kind of work day. And I work on weekends.
It’s definitely required a certain sacrifice of the normal, traditional, I guess, life. But again it’s just I love to do it, it’s not work to me, it’s really fun and exciting so I would just recommend just working if you want to, doing what you want to do. Also, to be productive, one thing that I like to do is actually doing all my planning offline. I find that having a computer is very distracting because I look at things like Reddit, Hacker News, and play some video games things like that.
What I actually like to do is take a notebook, sit outside and — it’s kind of like a waterfall project management thing — but I think of a feature that I want to implement, I sit there and I bullet point out tasks that I can do in on one sitting.
So I just break it down and do exactly what I can do on one sitting. And of course this isn’t a hard rule, as the requirements, you kind of change these things. But I then take that inside, as I sit down whether on my desk or on a plane or on a train, I just kind of look, choose one thing that I could do in a single chunk of time hopefully an hour or two max, finish it and then allow myself to you know, watch StarCraft or do some other distraction.
But as long as you finish all the bullet points, you actually finish something big and that’s kind of how that works. I worked on some pretty big features like the abstraction of Vagrant for multiple providers. It was an enormous refactor and it wouldn’t have been possible without breaking it down like that.
Alexis: Now you mentioned you do that feature-wise, do you also do that with your day as well?
Mitchell: No, my day is a lot more ad hoc, I think.
Alexis: A year or so ago you were on The Changelog and you mentioned Google Now thinks you work in Disneyland. Does Google now still think you still work at Disneyland?
Mitchell: No, I let my Disneyland pass expire.
Alexis: Oh, man!
Mitchell: It fell off, but now I want to go. But they don’t care. It’s a pro-tip, it’s an amazing place to work. A Disneyland pass cause is not cheap. It costs you like three hundred dollars something or four hundred now, I don’t know.
Alexis: It’s cheaper than a co-working space for you, that’s for sure.
Mitchell: It’s also far away from where I live. I got the Disneyland pass as a birthday present for my girlfriend then I got one for myself, because I mean there’s no point in getting one, and then we went a lot, we went in every two weeks or so. Then I started sitting at these cafes at Disneyland, and I was like, “You know, I think this would actually be a really pleasant place to pull out a laptop.” And I don’t carry around my laptop like that.
Then I kind of thought about it, and the next time I went to Disneyland I was with my girlfriend and I was like “Hey, I’m just going to see what this is like for like thirty minutes. Just indulge me and let me just check email for thirty minutes or something.” Because I had a MiFi, a Verizon MiFi and I just sat in California Adventure. I love this café restaurant place, it’s like a food court kind of, I don’t even know how to describe it, by this Monsters Inc. ride. I just sat there and pulled out my laptop, and no one ever comments, I got some looks like, “Why does this guy have a laptop?” But I mean I sat there for thirty minutes that first day without issue. Then I never worked in Disneyland for more than like two hours straight, but I definitely did but it was not a problem.
Alexis: I live in Orlando, so I’m somewhat tempted to do the same. But the cost of a Disney World ticket, or an annual pass at least, is pretty prohibitive by itself.
Mitchell: Disney World is pretty impressive. Disneyland is much smaller.
Alexis: So you’ve mentioned burnout before, where you had that three to six month period where you only focused on bugs and pull requests. How do you manage burnout now?
Mitchell: I don’t know. It’s actually a constant worry of mine because I don’t feel any sort of burn out tight now. I don’t really know what causes it. And I’ve read a lot about it. I don’t really know. Like I said, I work fourteen hour days, so I do kind of worry about it like, I guess will it just happen one day? Am I just going to be demotivated one day?
Alexis: When is the axe going to fall on you?
Mitchell: Yes. And this is very, very much a hypothesis — I have no evidence or anything to back it. My theory is kind of that, I’m working on a lot of different things now, from Vagrant to Terraform, and I’m also doing business stuff, so I live in excel spreadsheet for part of the day, and stuff like that. And I’m hoping that the diversity of everything kind of avoids that scenario, and I think I actually take more vacations than most people.
I probably go somewhere maybe like five or six times a year. I think that also probably helps. Vacation in that I might drive up a couple of hours for a weekend or something. Like I get away and disconnect a handful of times a year, more than the average person I think. I hope all this kind of comes together to avoid that.
Alexis: Alright. Speaking of that kind of thing, I guess this is a more similar feeling I have. How do you get a pass the fact that, when you take a break to play Star Craft or something for a while… do you ever have that feeling that, “Man, I really should be working on X or Y.” Do you suppress that kind of feeling? Or do you just ignore it as much as you can because you know you’ve already checked off those bullet points on that list you created during the day?
Mitchell: Yes, you know that’s actually my feeling all the time. It doesn’t matter what I am doing. It’s you know, I could be going to bed at 10. I go to bed pretty early because I’m more of a morning person. But I could be going to bed at ten and I think, “Man, I should really just answer those two emails again.” But I know that’s going to turn into like an hour somehow on the computer. It’s always a feeling. I think I’ve just gotten used to rationalizing with myself. “Oh, I did finish those two tasks, or I did send those emails, or I did take a look at that budget or something.” It’s like, “I could do this for an hour, I could not work for an hour is going to be okay.”
Alexis: The word is not going to end.
Mitchell: Yes, I mean yes definitely. Actually, my girlfriend is really good about it. We’ll go somewhere and I get fidgety if we’re doing something too long. And she could tell that I want to check email or I go back to the computer… It’s very unhealthy.
It’s really not a good thing. Someitmes I don’t like her for it but, she doesn’t give in and tells me to stop that. “What you do is not that important, and it can wait another hour. No matter how important you think it is, it’s just not.” It’s true. I mean it is true.
Alexis: I sympathize, man. So you’ve mentioned that it’s currently yourself and two others at HashiCorp, are you guys distributed? Or do you have an office you come into everyday or maybe once in a while?
Mitchell: We’re distributed. We’re going to keep the engineering team distributed for a little while. We’re hiring more on the business side now. That’s actually going to be centralized in on San Francisco. And you mentioned earlier I’m in Los Angeles. San Francisco and Los Angeles are 50-minutes away by airplane, fifty minutes and like $200 a round trip at most, that’s an expensive ticket. Probably $130 roundtrip. So it’s pretty inexpensive to fly there.
I fly there and stay there for about one to two weeks a month, you know it costs maybe like $150 dollars a month to do that. Istay at my co-founder’s house, so that’s free.
So, basically what I do, is I balance sometime between those two. And for engineers we just hang out in chat online and we just work remotely.
Alexis: Alright. So what one mistakes you’d rather not repeat in your journey so far with HashiCorp?
Mitchell: That’s a good question. There’s been some mistakes. I think, luckily, I haven’t made any fatal mistakes or else I wouldn’t be here.
Alexis: You’re still here.
Mitchell: But I think one mistake that I would avoid is going too slow. I think there’s sometimes where I’ve woken up and seen Hacker News or something and there’s something there I’d been working or had planned in the near future, but wasn’t just an idea, like I was actually about to do it. There’s been times where it doesn’t matter, where I’m happy that someone did it. But there’s also been times where I kind of kicked myself not doing something sooner.
I’ve done that for financial reasons and stuff like that but, to some extent, taking a little bit of venture capital and moving a little bit faster is not such a bad thing. And I would avoid going too slow for the sake of trying to be fully sustainable or something like that.
Alexis: On the flipside, what’s one decision that you’re particularly proud of which might not have seemed to be the correct choice?
Mitchell: I think we’ve done a good of job transitioning. We made this decision early on of transitioning HashiCorp, and it was named HashiCorp not Vagrant Inc. for this reason, but of transitioning it from the Vagrant company to kind of a more general ops company.
And I know a lot of people I talked to early on really questioned that. Because they were like, Vagrant is so popular, you should just like figure how to make money from that. And if I wanted to just make money away from Vagrant, I would have just taken one of those full time jobs that were offered to me. And I really wanted to do ops things but my fear was that people would see us too much of the Vagrant people and never fully embraced our other adventures to other spaces.
Armon and I made this decision early on, about eighteen months ago, that we were just going to focus purely on open source and ship as many open-source projects as we could for the next couple of years in order to build the foundation of the tools we had, and we were going to use wording in our talks, in our Tweets, and stuff like that, that kind of tried to push HashiCorp. as the vehicle rather than Vagrant and see what happens. And I think that with Consul and Terraform actually, we’ve started to see,the shift actually happening. Where people are actually are Tweeting now, “Oh like HashiCorp has another tool.” From the HashiCorp people there’s this. I think the branding switch has worked really well.
It hasn’t yet proven critical, but I’m still a firm believer, I’m really proud that we were able to do that. And I’m still a firm believer that that’s going to be extremely important in the next few years.
Alexis: You tweeted out the other day, “It’s shocking to me how my CS education comes into use daily. Thankfully I went through that process.” There’s a lot of developers that don’t have formal computer science training. Can you give us some examples how that’s come into play?
Mitchell: Well first, I want to just, I was going to say like, I don’t want to down play the people who don’t have CS background, but Jack who we hired doesn’t have a formal CS background and he’s of critical importance to the company. But for certain tasks, for certain architectural decision of software, having that CS background makes things so much easier.
Me and my co-founder, we went to undergrad together, so we joke all the time how much our education comes into play, and how everything is always a solved problem. We approach every problem and instead of — I’m just guessing here — but I feel as though people without CS background have no choice but to approach the problem with “How do I solve this?” and they try to solve it. Whereas people with the CS background, I’ve found, tend to approach problems, at least we do, with “How have others solve this before?”
Alexis: Right.
Mitchell: And how do we take that research and apply it to what we did. I actually think the biggest value of my CS research and CS background in addition to the core fundamentals was really having the appreciation for research. Being able to know how to read research papers, how to apply them. Someone asked in response to that tweet, “Give me an example.”
Alexis; You gave him three.
Mitchell: I got this text you can’t really pull emotion out of it, but I had this feeling that on of the people who asked that kind of had tongue in cheek, “Yeah, okay. You’re just saying that from your high throne because you have one.” You know things like that.
And I gave him three examples. In the past six weeks I had to use compiler knowledge, data structure knowledge and graph theory knowledge. That’s not abnormal. It’s stuff like that. Could someone without a CS degree have done it? Absolutely, no problem. But with the CS degree, I didn’t have to think about it, I just knew what the solution or at least the area of research that the solution was in, and we kind of just did it.
Alexis: Alright. I think we’re winding down here. What is your favorite text editor?
Mitchell: I use VIM. I used Emacs, until two years ago, and then I had so many VIM friends who gave me so much crap about Emacs. I loved Emacs I was fond of it. But I got so much crap about it that I was like, “Ok, well, I’ll switch to VIM for one month, thirty days. After the 30 days, no matter what decision I make, you guys can’t give me crap anymore.” And they said okay, and I used VIM and never switched back.
Alexis: So out of curiosity, the past folks that we’ve interviewed, which text editor do you think is in the lead?
Mitchell: In the lead? Actually I think VIM is very popular for the hardcore people. But I think in the lead, I would probably have to say is Sublime.
Alexis: Yes.
Mitchell: Yes. I think that, I mean, I recommend Sublime text to everyone. I honestly don’t believe that the effort to learn VIM or Emacs is actually worth it nowadays, but I don’t know, I did it. I don’t know why I did it. Can VIM be more powerful? Absolutely, but Sublime text is just good enough. It’s good enough.
Alexis: What are some open-source projects that you’re currently into.
Mitchell: This is kind of a horror answer, but I read the every single commit and code review for the Go programming language.
Alexis: Oh wow!
Mitchell: Yeah, I’m really into watching that thing evolve. I have no on a real level how the runtime works or I’m not in any position to contribute to Go other than the libraries. But I love watching the commits come in. I love watching the way legendary programmers like Rob Pyke and Russ Cox, and these folks I love watching how they work, what their code looks like, how they communicate on the mailing list, how they communicate in code reviews.
I just have this weird fascination with it that it’s almost like watching sports to me. I watch sports as well. Like, I’m into sports, but it’s almost like that. I watch it more with a sports-like view than a true understanding I guess.
Alexis: Learn from experience in a way.
Mitchell: Yes. Maybe some osmosis or something, but it’s just fun to watch.
Alexis: Is there anything that I’ve missed, that I should’ve ask?
Mitchell: No, I don’t think so. I think that was quite comprehensive.
Alexis: Where can folks go to find out about HashiCorp, Vagrant, and all the other projects you guys are working on?
Mitchell: Well, if the folks are really lazy they can just go to Google, and type in any sort of combination of that with contact information and they’ll probably get something. HashiCorp.com is probably the ingest point, there’s links to all the open-source projects from there. There’s also a contact email, but also my Twitter, my GitHub has my email. I don’t try to hide it in any way. I’m pretty easy to get in touch with. The company is pretty easy to get in touch with.
Alexis: Speaking of Twitter, where can we stalk you at?
Mitchell: MitchellH. That’s kind of the handle I use on every site that I feel matters.
Alexis: Alright, so MitchellH@yahoo.com, clearly? At Hotmail? Hotmail.
Mitchell: That’ll go to someone. It’s not me.
Alexis: Not you?
Mitchell: I actually don’t even have MitchellH on Gmail, someone else got it before I could, so that’s the only site that I don’t have MitchellH on, that I feel like I use on a day-to-day basis, so that’s sad.
Alexis: Alright and for us you can find us @Binpress. You find myself @alexissantos. Once again thanks Mitchell for carving time out of you schedule.
Mitchell: Yes, no problem, Thanks for having me.
Alexis: No problem. We will catch listeners again next week.
Mitchell: Alright, thank you.
Author: Alexis Santos