AWS EC2 Reserved Instance Auditing

If you’re running instances atop AWS, you’re probably at least bleeding money. If you don’t know what reserved instances are and/or aren’t using them, you are hemorrhaging money.

One of the worst parts of AWS billing, is that they don’t care to make it clear which instances are getting metered under a reservation, and which aren’t, so it’s near impossible to really tell at any given moment what your reserved instance coverage is. To that end, I wrote a small script that uses the AWS API to look at your reserved instances, and your running instances, and give you actionable information.

Continue reading

Posted in Architecture, Coding, Work | Tagged , | Leave a comment

Losing Aaron

Boston Magazine has published a very powerful interview with Bob Swartz, father of Aaron Swartz, nearing the anniversary of his death. I cried.

Posted in Life | Comments Off on Losing Aaron

Building Mapserver 6.4 and Mapcache 1.2 on Amazon Linux

 

More for me than anything else, but building and installing Mapserver and its cache from source can be a pain.

echo "/usr/local/lib" > /etc/ld.so.conf.d/xmaps.conf
yum install libpng-devel libjpeg-turbo-devel gdal-devel libxml2-devel geos-devel cairo-devel gd gd-devel giflib-devel mysql-devel freetype-devel fribidi-devel proj-devel fcgi-devel postgresql9-devel.x86_64 gd-progs libcurl-devel swig perl-libs perl-devel php-devel ruby-devel ruby-libs ruby-extlib java-1.7.0-openjdk java-1.7.0-openjdk-devel sqlite-devel pcre-devel apr-devel httpd-devel libtiff-devel libgeotiff-devel
yum remove java-1.6.0-openjdk
mkdir -p inst/mapserver-6.4.0/build
mkdir -p inst/mapcache-rel-1-2-0/build
cd inst/mapserver-6.4.0/build
cmake .. -DWITH_PHP=1 -DWITH_PERL=1 -DWITH_RUBY=0 -DWITH_JAVA=1
make
make install
cd ../../mapcache-rel-1-2-0/build
cmake .. -DWITH_TIFF=1 -DWITH_GEOTIFF=1 -DWITH_PCRE=1
make
make install

Oh, and you might want the ELGIS repo for some of those:

rpm -Uvh http://elgis.argeo.org/repos/6/elgis-release-6-6_0.noarch.rpm
Posted in Architecture, Linuxy, Work | Tagged , | Leave a comment

A[nother] Call To Arms

Bruce Schneier’s latest essay, essentially Battle For The Internet, is another well-defined look at the past, present, and possible futures given the current climate of pervasive government surveillance and over-politicization of what should be the neutral network.

This won’t be an easy period for us as we try to work these issues out. Historically, no shift in power has ever been easy. Corporations have turned our personal data into an enormous revenue generator, and they’re not going to back down. Neither will governments, who have harnessed that same data for their own purposes. But we have a duty to tackle this problem.

Who wins? Who wins.

Posted in Life, Opinions | Leave a comment

Adobe’s Creative Cloud Breached At Least Twice

So The Verge is reporting Adobe’s “Everything-Changing” “Creative Cloud” has been breached a couple times, dishing out all sorts of nice data about a few million users to the attackers. They fall back on the “don’t worry, your data was encrypted” but haven’t answered the nagging question whether their keys were better protected than their source code, which was pilfered… And in today’s world, we know how poorly non-open encryption fairs.

Get out of the clouds, people. Own. Your. Software.

Posted in General, Life, Opinions, Rants/Tirades | Leave a comment

CopyFS Update

Years ago I added features to CopyFS 1.0, then 1.0.1 came out and I never bothered to forward-port my changes. After spending some time moving, merging, rebasing, etc. in git this morning, CopyFS 1.3.1M is CopyFS 1.0.1 + CopyFS 1.3M.

I worked deliberately to ensure all of the old codes were left accessible, so if you check the tags, you’ll see each version (1.0 through 1.3.1M) available if you want to rollback or whatever. Have at. I don’t expect to be actively developing this, but pull requests are welcome.

Posted in Coding, Linuxy | Tagged , , , | 2 Comments

Ruby Hash Extensions

As I mentioned before, one of the nice things about Ruby is its malleability. Here are a couple extensions to the Ruby Hash object:

class ::Hash
  def random
    return self.keys[rand(self.size)]
  end

  def byValue(subKey)
      if subKey
        self.sort_by {|key, value| value[subKey]}.reverse
      else
        self.sort_by {|key, value| value}.reverse
      end
  end
end

Simply, Hash.random will now return a random hash key. Hash.byValue will sort the hash by value. Hash.byValue(‘priority’) will sort a Hash-of-Hashes by the value of the named sub-key. TIMTOWTDI, for sure.

Posted in Coding, Linuxy | Tagged | Leave a comment

Ruby 2 From Source on CentOS 6.4

Ruby‘s autoconf doesn’t alert you to the fact that your system doesn’t have everything it could use. It’ll blindly clear you even though, for example, nothing crypto-related will work. On a fresh CentOS 6.4 install (with EPEL) the following will give you everything you need to build like a champ:

yum install gcc make gcc-c++ openssl-devel libffi-devel \
ncurses-devel gdbm-devel readline-devel bison-devel bison
Posted in Coding, Linuxy, Work | Tagged , , | 2 Comments

Out Of The Clouds

Dear Everyone:

You’re understandably upset that some of the world’s governments have been and will continue to spy on you. This is your fault. You kept shipping your data to “the cloud”. You kept outsourcing your local services to “cloud services”. You “outsourced” your IT functions. You embraced GeoCities, and then MySpace, and then FaceBook, and now Twitter.

If you don’t like this, then stop. Get out of the clouds. Locate your data within your control. Locate your services with your business. Accept that “the Internet” is the threat vector as we used to, and secure against it.

Yours, in decentralizing,
M@

Posted in Life, Opinions, Rants/Tirades, Work | 2 Comments

Which Oar Best Rows The Boat

Ted Dzuiba wrote a post that boiled down to identifying his own “language bigotry” as a step along the way to software engineering “Mastery”.

He’s absolutely correct about introspecting why one chooses to “fight” other languages. And by saying “languages” I could substitute OS platform, cloud computing, database manager, code editor, car manufacturer, religion,  Constitutional Rights, regional sports team, patriotism, etc.

We identify ourselves by our passions and beliefs.  As a means of vetting prospective employees, I have long made a habit of asking candidates “what are you?” The answers over the years include things like “mom”, “Catholic”, “firefighter”, “LARPer”, “Mac geek”, “soldier”, “Cisco network engineer”, “hunter”, “professional juggler”, “Red Sox fan”, “clergy”, “MySQL admin”, “mountain climber”, “Chevy nut”, “Java programmer”, and on and on.  When someone identifies themselves as a something you can pretty safely assume some other things: A “Red Sox fan” most likely loves baseball, hates the Yankees, also likes the Patriots [American] football team, and is located or grew up in the New England region. A “hunter” most likely feels strongly about their Second Constitutional Amendment Rights, votes centrist or right on the political scale, dislikes leftists, also enjoys the outdoors, probably fishing, and most likely desires to live somewhere more rural than more urban. Everything there is a prejudice for or against something, and it’s perfectly natural. (And yes, it’s stereotyping: that’s how we templatize correlated attributes, please carry on)

The key to finding a good employee is sussing out which of those will get in the way of delivering business value. The key to being a good employee, is introspection about why you have those feelings and if they’re relevant. As Ted says perfectly (I emboldened his italics):

I feel orders of magnitude more useful delivering business value than I feel delivering code.

A clutch statement right there – it’s not about the tools, it’s about the product and its value. I’m not advocating, and I doubt Ted is either, that any intelligent engineer should just clam up and do as they’re told – if there is value or purpose to moving against the grain, then speak up! Your value as an engineer is only partially what is produced, and constructive engagement debating how something is produced is exceptionally important as well. But know when your beliefs- your prejudices- are pointlessly in the way. Introspect on why.

My most recent position requires the use of an Apple Macintosh (gross), is completely hosted using Amazon Web Services (ick), systems scripts almost entirely written in Ruby (*twitch*): all of which I knew in advance of accepting the position. I don’t prefer a single one of those things, but would it have been a better position if they used HP laptops running Linux, had a hardware server farm, and churned out klocs of Perl? Nope. Would it be a better business? Not at all. So I use a Mac for hours a day, architect in concert with AWS’s strengths and weaknesses for hours a day, write Ruby for automation as needed, and am perfectly at peace with all of that.

Sure, I still run Linux on my own hardware, remind my colleagues that if we ran our databases on hardware vs. “the cloud” we wouldn’t have “that” problem (for whatever the problem might be), and occasionally sneak in a quick Perl script that would have taken three-to-six-times as long for me to write in Ruby (not necessarily Ruby’s fault) – but not at the cost of value.

A bad workman complains about his tools. A craftsman works with what is at hand.

Posted in Architecture, Coding, Life, Opinions | Tagged , , , , | Leave a comment