Dr. Nic Williams talks to Mitchell Hashimoto about Vagrant.
- 0:20 All about Mitchell
- 3:45 The OSS grant: Vagrant
- 6:00 How Vagrant works
- 7:15 Networking options for Vagrant
- 8:45 Who uses Vagrant?
- 12:00 When should you use Vagrant?
- 12:45 Fog and Vagrant
- 18:25 Kiip
Links About Mitchell Hashimoto
\"Twitter\":https://twitter.com/mitchellh
\"Website\":http://mitchellhashimoto.com/
\"Github\":https://github.com/mitchellh
\"Vagrant\":http://vimeo.com/9976342
Dr Nic: Welcome to Cloud Out Loud podcast, I am ably and joyfully joined today by Mitchell Hashimoto. Mitchell, welcome.
Mitchell Hashimoto: Thank you, thanks for having me.
Dr. Nic: Now, there’s only one thing I wanna talk about which is Vagrant.
Mitchell Hashimoto: Okay, I guess I’m qualified to talk about that.
Dr. Nic: You’re qualified but I think I’m pretty confident that the story behind Vagrant starts before Vagrant. So what were you doing before Vagrant?
Mitchell Hashimoto: Right before Vagrant I was working for a Rails development shop working on pretty cool projects but seeing new projects every six weeks, six to ten weeks and I was getting frustrated with setting up new Rails versions, different web servers, different queue servers, databases, all that stuff associated with new projects so I wanted to find a way to package it all up into one thing that I could bring up and tear down all at once and virtualization happened to be the answer to that.
Dr. Nic: So was – was there no like AWS back then?
Mitchell Hashimoto: There was AWS. Actually so before Vagrant I wasn’t good with servers at all, like I didn’t know – I didn’t know –
Dr. Nic: So you wanted like a quiet, private place to do this where no one was looking.
Mitchell. Yeah.
Dr. Nic: The evil overlords of AWS.
Mitchell Hashimoto: Yeah, I probably wasn’t even able to set up a like production Rails server back then, I’m sure I wasn’t, I know I wasn’t so. I just – I just wanted a sandbox to mess up.
Dr. Nic: I’m still unsure if I wasn’t working at Engine Yard and access to everything that we have whether I could do it.
Mitchell Hashimoto: Yeah, I don’t know, I don’t know.
Dr. Nic: The Rails command I’ve got.
Mitchell Hashimoto: Rails, new or in it or whatever it is now.
Dr. Nic: It’s Rails server right?
Mitchell Hashimoto: I mean creating a new project, not to launch a server so.
Dr. Nic: We trust our customers to come with an app of some sort, we can find you one, we’ve got some lying around.
Mitchell Hashimoto: No, that’s okay.
Dr. Nic: Was VirtualBox it at the time, I mean there’s other VM’s.
Mitchell Hashimoto: Yeah, I looked around. At the time I was a big fan. I mean I paid parallels and I had parallels so.
Dr. Nic: That was the big one on the Mac wasn’t it, it was parallels?
Mitchell Hashimoto: Yeah, at the time I think so. Fusion was coming – was an up and comer at the time but –
Dr. Nic: Because there were a lot of parallels jokes. It was like what’s the best – the best place to put Windows in is a window. That was – I’m not sure – I didn’t think that was formal parallels marketing material but there are certainly – it had a lot of buzz around that.
Mitchell Hashimoto: I didn’t hear that but – that’s funny. Yeah, I looked at – I was looking around but none of them had API’s so VirtualBox was actually my last hope and I took a look at it and it happened to have a fully, complete API so I went with it.
Dr. Nic: It still – the website still looks like someone’s last hope.
Mitchell Hashimoto: Yeah, it – yeah, it does. If you see the UI it’s still kind of a last hope.
Dr. Nic: I mean in the time that I’ve known you and since I’ve known that – and you look back at the old sites that come from Sun to Oracle, they’ve touched the logo, they’ve changed the logo from Sun to Oracle and I think that’s it.
Mitchell Hashimoto: Yeah, the sad thing is they touched the logo but they never changed like the font colors or anything so all the colors are still sun colors but Oracle’s – Oracle’s red so nothing’s red yet, it’s funny, it’s all blue.
Dr. Nic: It doesn’t have that – it doesn’t have that white, gray, red theme that makes you just not wanna use an Oracle product.
Mitchell Hashimoto: Yeah, no.
Dr. Nic: That’s good. Which actually is interesting because just to start a quick conversation one of the things a year ago you and I, we caught up and we did the OSS grant around where we started that with Vagrant. One of the things we were working on at the time was libvirt. What was that?
Mitchell Hashimoto: I had, at the time I thought that – so libvirt advertised itself as one API to many hypervisors, basically like the ActiveRecord for Fog –
Dr. Nic: The Fog for high-class computing.
Mitchell Hashimoto: Yeah, the Fog for hypervisors. And that sounded pretty good so I wanted write a wraparound that so Vagrant could work with--
Dr. Nic: Because you know they’re not exactly the same but you wanted – you wanted to sort of –
Mitchell Hashimoto: Yeah, I thought that it would just kind of make it work for me. But it turns out – libvirt’s great but it turns out that you do have to still have to know the quirks of each hypervisor and kind of like create a template for each of them so in the end it doesn’t buy you much in terms of what I was using it for. So I’ve since stopped that because it wasn’t gonna work.
Dr. Nic: Okay, so that was – that whole notion of hypervise and VM’s is obviously the basis of cloud computing at the moment. I think there’s one cloud computing company that is shipping dedicated hardware. I think UService; they’re like API driven – Rental of machines and you actually get real machines.
Mitchell Hashimoto: Yep, yeah.
Dr. Nic: The rest of them are all – what are the different VM systems?
Mitchell Hashimoto: For production? They’re probably Zen or KBM are probably the only two that you hear of.
Dr. Nic: They’re the two open source one, what’s the one – what’s the VM ware one?
Mitchell Hashimoto: Oh, the vSphere?
Dr. Nic: There’s some – yeah, there’s some companies that make business out of selling virtualization but it’s obviously big business. So how similar, once your inside VirtualBox instance or Vagrant image – actually firstly what’s the different being inside VirtualBox and being inside Vagrant?
Mitchell Hashimoto: I mean it’s the exact same. Vagrant’s mostly just a management – a command line management interface to manage all the complexities that virtual box has so –
Dr. Nic: It has its own API command line.
Mitchell Hashimoto: VirtualBox? It does but you’d have to do everything manual so if you wanted to import a virtual machine that’s one command for each but if you want it to do networking then there’s not only the VirtualBox command you have to give and you have to configure all the network interfaces correctly but you also have to do some work in the guest operating system itself.
Dr. Nic: Okay, so let’s go through it. So we know I type Vagrant Up – I don’t wanna sound like the ignorant interviewer as best as possible.
Mitchell Hashimoto: Nic knows how to do this.
Dr. Nic: So I go into a new project, right? I go Vagrant and I correct to Vagrant file with some standard – some template stuff, it’s gonna give you a local IP address; it’s going to give you SSH enabled – go through that sequence. What – what are all the things that happen?
Mitchell Hashimoto: So when you type Vagrant Up basically on the most basic form without any configuration what it does is imports the disc image, sets up port forwarding so you get SSH into it, set up the meta data for shared folders so that your current directory gets shared onto the virtual machine so you can continue editing files on your own computer. And then boots it and verifies that it is SSH-able for you. And that’s what it does in the bare minimum.
Dr. Nic: And it runs on a – on the local host – on the local network but under a different port number but you can do the post only –
Mitchell Hashimoto: Yeah, so there’s two networking options for Vagrant. It’s forwarded ports which open up your – which basically foreign say port 22 on the guest to something else like 22 22 on the host and that actually opens up your public port too so someone else could hit that port on your machine. Then there’s also host only networking which creates a private network between your machine and virtual machines that you have and that’s not accessible externally at all but allows you to mirror – mimic clusters and do other neat things.
Dr. Nic: Now I like the – so the default – the default IP address that you give 33 33 33 10 is the –
Mitchell Hashimoto: It’s the Department of Defense IP so when I was first doing –
Dr. Nic: You decided that of all the potential customers that might use it the Department of Defense are last on the list and you’re just gonna use their IP address range.
Mitchell Hashimoto: Yeah, the problem was if you use – so I’ve gotten kind of some negative flak about this it’s because people will say well, there’s an RFC that tells there’s these reserved IP’s that you’re supposed to use but the problem is if you choose an IP that happens to conflict with your router or with a VPN connection that you’re on or something then the VirtualBox networking just breaks. And there’s really nothing I could do to figure that out easily so I just advertised an IP that is not RFC compliant but also is probably not gonna be used.
Dr. Nic: It’s a lot funnier.
Mitchell Hashimoto: Yeah, it’s a lot funnier. It’s one of the like 10 million IP addresses that the Department of Defense owns so.
Dr. Nic: It’s – it’s – right, and I’m sure they’re all funny. So okay so Vagrant exists, you can build a VM, who uses it and what are the different types of users?
Mitchell Hashimoto: The users go from individuals building their own personal websites to teams of small companies to hundreds of employees at larger companies. So some of the larger companies I guess would be like Eventbrite has around 200 employees, they use is for all their development. Nokia in Berlin, basically all Nokia’s European division uses it to test infrastructure. LivingSocial uses it to some extent so there’s big companies out there using and it there’s also small people so it’s neat that it’s a tool that scales up and scales down.
Dr. Nic: Now I heard that Travis CI is using it.
Mitchell Hashimoto: Yeah, Travis is using it for CI so when you push a project to Travis they actually isolate your project in a Vagrant instance so that you don’t – there’s no security risks at all involved with running other people’s codes.
Dr. Nic: So are they running on hardware or are they running in another virtual environment?
Mitchell Hashimoto: I don’t actually know what they’re server setup is like, I’ve talked to them a few times to help them with problems but I don’t know what their exact setup is.
Dr. Nic: Gotcha. All right, so if I pretend to not understand for the sake of this interview – so Vagrant is a piece of Ruby code that sits on – ultimately on top of VirtualBox, an article branded delicious piece of open source. It’s a stack of technology within guest OS’s being built. What – does that just always work, is it a fragile or is it at the other end of the scale and it’s less fragile?
Mitchell Hashimoto: So I guess the general – you’re asking about the general architecture of Vagrant?
Dr. Nic: Right. I mean am I constantly gonna have issues because of these moving parts, do I assume –
Mitchell Hashimoto: Got it, got it. So there’s – so Vagrant is a piece of Ruby code on top of VirtualBox. The interface between VirtualBox and Vagrant is just the CAPI which is – with each Vagrant release I say which version of VirtualBox it works with and that’s – that will be stable, that’ll be fine. And then on top of that there are more fragile pieces but they’re not likely to break because they’re – because they change, they never change but if you try – for example each mounting folders is different on each operating system so there’s actually logic in Vagrant to know how to mount a folder in OSX versus FreeBSD versus Ubuntu, Debian or even Windows.
Dr. Nic: As the parent. What do you call the guest-side relations inside the VM, what’s the outside VM?
Mitchell Hashimoto: The outside I call it the host.
Dr. Nic: The host. So you’re talking about the host in that case.
Mitchell Hashimoto: Yeah, the host needs to know – the host – there’s some host specific things like creating NSF experts and then there’s some guest specific things like mounting folders, creating new network interfaces and so on. And they’re not – they’re not fragile in the sense that things could change but if you try to use an OS that’s never been used with Vagrant before, then you’re likely to be missing more advanced features but the basic will always work because it only requires SSH.
Dr. Nic: Very cool. So what can’t I do with Vagrant? What’s an example of – I mean it’s interesting to explore a premise. I think it’s interesting. Where to use Vagrant or where to use other cloud computing or some other comparable thing?
Mitchell Hashimoto: Something people tried to do early on with Vagrant that still doesn’t quite work properly because I didn’t design it for this was deploy production systems, they – some group of people saw Vagrant as a replacement almost for Cap, like they thought that they’d be able to Vagrant Up from their development machine and it would actually spin up like an EC2 instance for production and that’s not really a good use case because it’s not what I made it for although it could probably work that way. Other use cases I guess, it’s not very good for –
Dr. Nic: That’s right, you and Wes a year ago sort of had discussions around what it meant for Fog and Vagrant to work together. Somehow those – there seem to be something meaningful there and I know Fog itself mentions VirtualBox, I don’t think it mentions Vagrant per se but it references. I know I can turn it off stuff with Fog, I think that’s about it.
Mitchell Hashimoto: Yeah, I still – I still think there’s a place for Fog with Vagrant, we’ll see.
Dr. Nic: Well, yeah, whether it’s worth the effort.
Mitchell Hashimoto: Yeah, yeah.
Dr. Nic: And then there’s – the guest OS itself, I mean can I take a – both directions, can I take a Vagrant image import it up into a Rackspace or Amazon’s universe or?
Mitchell Hashimoto: Okay, so Vagrant has a package feature and it will allow you to export VirtualBox images but it exports it in VirtualBox’s format which is only a format VirtualBox can export. And there are guides online for how to convert like a VirtualBox image to an AMI for Amazon or something like that so it’s certainly possible but Vagrant doesn’t do much to help you.
Dr. Nic: So a better path might be that you build – you’d start with a scratch it if you’re gonna use an Ubuntu or you’d use an Ubuntu VirtualBox image, you’d take an equivalent Ubuntu AMI port and then you’d shift because you have all the provisioning mechanisms. At what – what are my options for taking a raw – a raw image and turning it into something?
Mitchell Hashimoto: So there’s a few options you could use. Basic shell scripts if you don’t – that’s the minimal, you don’t want to learn any tools or anything, you just write shell scripts. You could use Chef, either Chef Solo or Chef Client or the Chef server or you could use Puppet, both the server and the stand-alone version. But it’s also – it’s just a plugin interface so you could plugin any Ruby code you want so if you have your own built in – your own homemade computer –
Dr. Nic: I think Jason from the office was playing with – as a provision of the SM framework, I’m not sure what that looks like. I haven’t seen you – I don’t know if he didn’t push it up or he hasn’t mainlined it yet.
Mitchell Hashimoto: I haven’t seen it yet but yeah – so yeah, you could plugin anything you want but out of the box it covers 99 percent of the cases.
Dr. Nic: Well, I mean Shell –
Mitchell Hashimoto: Shell, you got it.
Dr. Nic: Allows –
Mitchell Hashimoto: You could do anything.
Dr. Nic: Pretty flexible. Well, can it be interactive? Obviously it starts interactive with Vagrant Up, can the provisioner mechanism allow for question and answer or run –
Mitchell Hashimoto: It cannot, it’s all run – it’s basically –
Dr. Nic: It’s unlike Capistrano where you can scrape the things and ask questions and send answers back and.
Mitchell Hashimoto: Right, it – none of the built in ones could but technically it could be if someone wanted to.
Dr. Nic: Really wanted to if it was that important.
Mitchell Hashimoto: If it was that important.
Dr. Nic: Well, actually so I’m thinking dev, there was – there was a user of Vagrant a year ago, Dev Structure--
Mitchell Hashimoto: Dev Structure, yeah.
Dr. Nic: And they’re – certainly they’re Vagrant Up asked question but that must have been external above the actual mechanism provisioning, it was probably collected information, I can’t remember.
Mitchell Hashimoto: They did a lot of cool things in Vagrant. I don’t remember exactly.
Dr. Nic: And what happened to them?
Mitchell Hashimoto: They just kinda switched products that they were working on and then Vagrant didn’t fit in as neatly as they –
Dr. Nic: I know it’s kind of interesting what they – what they were doing, yeah, with – because you could describe and then it would ship up to their production.
Mitchell Hashimoto: Yep.
Dr. Nic: So you mentioned that this is all done with the CAPI which is always interesting to therefore discuss as to what that has meant for you living – because VirtualBox is – the Windows thing is a Unix thing, it’s a OSX thing, has that made Vagrant easy to work with in all those things?
Mitchell Hashimoto: No, not at all. I’ve always wanted Vagrant to work identically on all those platforms and it’s been very difficult because on Linux, Linux’s has always kind of been the first class citizen of Vagrant because that’s what I use but Windows has always worked but it’s always trailed a little bit behind and it’s mostly because the CAPI doesn’t exist on Windows and you have to use the COM API and Ruby standard library com doesn’t work.
Dr. Nic: Yeah, I remember when you actually wrote a blog post about the consequences of following this because if you’re on 60 – if I remember correctly, if you’re on a 64-bit Windows machine therefore the VirtualBox commoner face is 64-bit. Not surprisingly the Ruby Win32OLE library –
Mitchell Hashimoto: That’s Win 32 –
Dr. Nic: Does Windows support for OLE for something to do with the 32.
Mitchell Hashimoto: Yep, yep, so it doesn’t work at all in 64-bit so you have to use JRuby which does work.
Dr. Nic: Which at the end of last year had just actually added, ironically they’re called Win32 as well.
Mitchell Hashimoto: It’s called Win32.
Dr. Nic: Even though it’s more – yeah, that’s interesting so we ended up with yeah – that was the start shipping you builds, it was around the one – JRuby 1.6, starting getting new Windows 64-bit built.
Mitchell Hashimoto: Yep, unfortunately that’s still not perfect though like the standard library Ruby one really has no way to handle output parameters that are a raise so even though JRuby – JRuby’s great and they wanna make it as compatible as possible they don’t know how to make it compatible because even C Ruby has no way to do it. So there’s certain features in Vagrant that I just cannot get to work on Windows.
Dr. Nic: Can’t you just run VirtualBox in 32-bit mode?
Mitchell Hashimoto: Yeah, you can but it’s – you’d have to specifically install it as such and it’s not – I mean it’s very easy for the user to make a mistake of installing 64-bit.
Dr. Nic: Well, I mean sorry, if you just – can you just – Windows just has a 32-bit and a 64-bit download or –
Mitchell Hashimoto: VirtualBox does, yeah.
Dr. Nic: What did I say? No, so VirtualBox. So basic on the VirtualBox site you get – you want the 32-bit version, you wanna install that.
Mitchell Hashimoto: You could, I recommend JRuby on Windows, that’s just more predictable. For right now it’s a better route to go. The 32-bit Windows one, 32OLE I don’t trust it as much.
Dr. Nic: Trust. Mitchell, is there any other topic I should ask? We could talk about the shiny badge you’ve got on your laptop.
Mitchell Hashimoto: The badge, this thing?
Dr. Nic: Yeah.
Mitchell Hashimoto: No, that –
Dr. Nic: Which is Kiip, K-I-I-P.
Mitchell Hashimoto: Kiip, yeah, that’s my employer.
Dr. Nic: That’s your employer?
Mitchell Hashimoto: Yep, we do –
Dr. Nic: What do they do?
Mitchell Hashimoto: We do real awards for virtual achievements, just another take on a form of advertising so instead of carving out like banner space like the web does for mobile devices we’ll find achievements and then those achievements will give people rewards. So instead – when you get 1,000 points in a game you win a free coffee or something. And because of that you’re more excited, you get rewarded for doing something cool and brands get a real engagement with whoever’s playing.
Dr. Nic: Very cool.
Mitchell Hashimoto: It’s fun.
Dr. Nic: Mitchell, thank you very much for coming on Cloud Out Loud.
Mitchell Hashimoto: Yeah, thank you.
