I had my doubts about Mailgun, because there was no suggested Mailgun
library. Instead, the docs recommended developers just install
Requests. I expected to
write some boilerplate as a result, but they proved me wrong.
Check out the Mailgun docs for sending an
Examples in 6 languages and curl, and the examples update with your
API endpoints and auth data if you’re logged in. That’s some
documentation, and I’m now a huge fan.
If you want to see how we tie the systems together, check out the
code. I’d like to morph that short script into a small web application
that will guide developers through the ATC process. Reach out if you’d
like to help!
Back on the topic of the training itself: we’re always looking for
more in-class mentors and assistants! If you’ll be in Tokyo on
October 25th and 26th, please consider registering as an
Each base image type,
defined in Nodepool’s YAML configuration file,
includes a setup attribute that matches one of the scripts. The
matching script is executed in an environment that includes any
NODEPOOL_ variables present in the Nodepol daemon’s environment. From
all that I’ve seen, this typically only includes NODEPOOL_SSH_KEY.
So to build replica nodes for personal use, I should just need to copy
the scripts to /opt/nodepool-scripts and execute the right one in my
packer provisioning configuration.
Remember the agility in the last presentation? Part of that
includes the platform itself: OpenStack releases every 6 months.
No cloud platform is simple: lots of moving parts.
OpenStack runs a sophisticated continuous integration system with Jenkins
eNovance runs a duplicate CI infrastructure chained from the OpenStack CI system.
eNovance also runs a duplicate CI infrastructure chained from their
corporate CI systems in customer cites.
They’ve built tooling to allow progressive upgrades and rollbacks of
installed clouds: rack by rack, for instance.
They often run as little as two weeks behind the stable branch.
As a counterpoint to this, I met with someone who was struggling with
an upgrade for over 6 months. They installed OpenStack’s Diablo
release years ago, and they’ve given up on the idea of smoothly
migrating to Havana (released in November of 2013). Instead, they’re
dropping in a new cloud on new hardware. If you don’t stay up to
date, you’ll get into a treadmill of invasive installations and migrations.
I’ve used open source for longer than I care to remember, and I’ve
even submitted a few patches along the way, but I’ve never become a
member of a project. I felt a little under- and over-qualified
simultaneously. How could I feel overqualified if I had never become
a repeat contributor? I know how to program, but contribution is a
lot more than programming. See if any of the phrases sound familiar:
I just need to sit down and figure it out. (repeated weekly)
I’m going to work on bug $X tomorrow. (repeated weekly)
I hope I do this right, I don’t want to look stupid.
Here’s a hint: “I just need to sit down and figure it out” repeated
more than once is code for “I don’t really want to do this” or “I have
no idea where to start, and I could use a little guidance.” In my
case, it was the latter.
The size and sophistication of OpenStack intimidated me. I’m no
stranger to code review, continuous integration, and automated
testing, but OpenStack is big. There are more hosts running or ready
to run automated tests for OpenStack than we use to service our
biggest application at $DAYJOB. (Check out the “Job Stats” graph on
the Zuul Status Page to see a live
count of hosts.)
My biggest problem was a people problem: I was convinced that while
navigating the OpenStack build and review environment for the first
time I’d make a mistake, and people would think I’m stupid. In
retrospect, that was stupid of me. Of course I’d make mistakes, but
the people and the processes exist to help us build a better product,
accounting for the possibility of mistakes along the way. OpenStack
doesn’t get built in large chunks of caffeine fueled code tossed over
a wall. We build it one small, well reviewed, well tested patch at a time.
The OpenStack Upstream Training broke down into three major components:
The Training Class
The Design Summit
The Training Class
Scheduled in the two days prior to the Summit, we soaked in the
technical and social processes of OpenStack, and we didn’t have time
forget it all before the Summit began.
The bulk of our classroom training focused on the people and the
social processes in open source, and specifically OpenStack. Behind
every patch and every review comment there is a human being with
hopes, goals, stress, problems, and dreams. We all want to build a
great system, and we’re all constrained to 168 hours each week. The
PTL that doesn’t respond to an email isn’t ignoring you on purpose,
she’s drowning in email and constantly trying to dig out. Gerald
Weinberg reminds us in his writing that “no matter how it looks,
everyone is trying to be helpful.” We can never go wrong with empathy.
Summits are more than just a traditional conference. Design
discussions to shape the next release fill more than half of the week.
Upstream students’ most important assignment for Summit week was to
find and meet the team members that would review our chosen work.
“Introduce yourself, tell them what you’re interested in working on,
and how much time you will commit each week. They are busy, they will
I met a large chunk of the Designate team, and I learned how little I
know about running DNS at scale. That’s OK, though, I’ll learn more
as I go, and the team will help along the way.
The class followed by the summit made for an exhausting week. It
would have been easy to fly home, sleep for a day, and get lost in
$DAYJOB work for a week trying to catch up. “I’ll do some OpenStack
work next week once I’ve caught up on home and work.”
Wrong. The habit of OpenStack contribution isn’t yet built, and
taking a week off will destroy any momentum. The program includes
weekly mentoring sessions to keep students moving forward.
Loic set aside valuable time on Wednesdays and
Saturdays to check in on students and keep them moving toward their
goal, and it was vital for my success. I stayed up until 4:00 AM the
night before my first mentoring session because I didn’t want to let
him down, and I wasn’t happy with my progress thus far.
I now have one patch merged, one under review, and one in work. I’m
also working on improving the automated testing for Designate, and I
suspect I’ll work on documentation in coming months as well.
Should you attend?
You, too, should attend OpenStack Upstream Training if you feel
capable but stuck when considering open source contribution. The
training will give you the social and process knowledge to succeed,
and the encouragement to keep moving forward until you build up your
own momentum. It’s hard work, but it’s fun work.
The Upstream University training changed the trajectory of my career.
I’ll forgive you for dismissing that as hyperbole since I’m still so
early in the process, but I already feel addicted to the scale of
OpenStack and the hum of hundreds of sharp developers and operators
improving it one patch at a time.