Articles

TransitStopMotion.rb

In Hamilton, Ruby on December 4, 2013 by Matt Grande Tagged: , , , ,

Over the past couple days, in the evenings, I’ve been working on a small project.

Frequent readers of my blog (ha!) know that I enjoy map making and things of that nature. I decided to do a more local project, this time, using Hamilton’s transit data.

All the routes

The script I’ve created takes a transit schedule (in GTFS format) and either places all the routes on one image, or creates thousands of images and turns them into frames in a video.

Here’s a quick overview of how it works:

  1. Determine “significant” trips (in this case, I’m only looking at weekday trips).
  2. Generate a list of stops with their times and latitudinal / longitudinal co-odrdinates.
  3. Convert the lat/long into x/y.
  4. Plot to an image.

If you want to take a deeper look into the code, here it is on github. If you want to see the image again, feel free to click here. If you’re all about the videos, maybe give this link a click?

Articles

Combining and minifying files (in order) using Shell Scripts

In shell, Uncategorized on October 10, 2013 by Matt Grande Tagged: , , , , ,

I’ve been working on an application using Backbone.js recently. I really enjoy it, however I realised today that I now have over 100 javascript files in my project. This means that every time my page loads, the browser is making 100 separate HTTP requests. Very quickly, your site slows down.

The obvious solution is to combine all my scripts into one file and minify. I’ve done this several times before, but I needed to make sure that certain files loaded in a certain order, so I whipped up a shell script to ensure some of the files are pushed to the front of the line.

Having never really done that much shell scripting before, this wasn’t the easiest task for me. To help save others some time, I’ve attached my script and documented it pretty well. Let me know if you have any questions!

screenshot

Above, you’ll see our (sample) folder structure. Under the models subfolder, we have tab.js, which needs to be loaded before subtab.js.

Below, you’ll see the gist of what I’m doing. If you have any sort of shell experience, I’m sure this isn’t a revelation, but I wish I had something like this when I started writing earlier today!

A few things to note:

  1. I highly recommend adding this code as a git hook, if you want your combined JS to be in source control
  2. The heavy lifting is done with the fabulous YUI Compressor, which requires Java to be installed (It also does CSS minification).
  3. Remember to chmod your compression script to allow you to execute or else it won’t work!

Articles

BitBucket -> GitHub migration

In Uncategorized on October 9, 2013 by Matt Grande Tagged: , , ,

The Move

Many moons ago I started releasing some of my projects out into the wild. I used BitBucket, rather that GitHub, for a few reasons. One of the biggest ones was how frustrating Git was as a Windows user when I first tried it. Coming from SVN, Hg was a much gentler introduction to the world of DVCS (thanks in large part to Joel Spolsky’s HgInit).

Now, several years later, Git is as easy to use as anything else and GitHub has a much larger community that BitBucket.

The long and short of it is, if you want the latest version of anything I’ve done, check me out on GitHub.

Other News

HamOntFire is no longer working, due to Twitter removing v1 of their API in favour of the more secure (but less open) v1.1 API. I plan on upgrading the site, but I don’t currently have time or motivation. Hopefully sometime in early 2014.

I’m currently working on another project that mixes my three greatest loves: Ruby, Rugby, and statistics. That will be done in time for the Women’s Rugby World Cup in 2014. I have a (basic) design, a database full of results, and a stats calculator built; now I just need to put them all together.

Articles

All the roads in Canada

In .Net on May 6, 2013 by Matt Grande Tagged: , , ,

If you’re just here for the image, click here (10638×9101, 1.17MB). If you want more information, please read on!

I recently saw a map of Oregon containing only the roads, and nothing more.  I thought it was really interesting how you could see where the population centres and natural features (lakes, mountains, etc) were.  I decided to try something with Canada.

The first challenge was finding a dataset containing all the roads in Canada. Luckily, it was on StatsCanada’s website, but it took longer than expected to find.

I opted for the GML format, since it’s the one I’m most familiar with.  The full file is a fairly large 1.9GB.  Here’s a sample of one road, Wilcox Street in Hamilton, Ontario:

<gml:featureMember>
<fme:lrnf000r12g_e gml:id="id00987e75-8fb2-4224-8e82-f2b97dd86307">
<fme:NGD_UID>4677289</fme:NGD_UID>
<fme:NAME>Wilcox</fme:NAME>
<fme:TYPE>ST</fme:TYPE>
<fme:CSDUID_L>3525005</fme:CSDUID_L>
<fme:CSDNAME_L>Hamilton</fme:CSDNAME_L>
<fme:CSDTYPE_L>C</fme:CSDTYPE_L>
<fme:CSDUID_R>3525005</fme:CSDUID_R>
<fme:CSDNAME_R>Hamilton</fme:CSDNAME_R>
<fme:CSDTYPE_R>C</fme:CSDTYPE_R>
<fme:PRUID_L>35</fme:PRUID_L>
<fme:PRNAME_L>Ontario</fme:PRNAME_L>
<fme:PRUID_R>35</fme:PRUID_R>
<fme:PRNAME_R>Ontario</fme:PRNAME_R>
<fme:CLASS>23</fme:CLASS>
<gml:curveProperty>
<gml:LineString srsName="_FME_0" srsDimension="2">
<gml:posList>7197110.1 878145.377099998 7197136.3143 878156.274300002</gml:posList>
</gml:LineString>
</gml:curveProperty>
</fme:lrnf000r12g_e>
</gml:featureMember>

The most important piece of information in there, for my purposes, is the <gml:posList> tag.  In this case, the data is stored as pairs of Eastings & Northings.  This means that for the above road, it travels from 7197,110 metres East and 878,145 metres North of the origin to 7,197,136 metres East and 878,156 metres North of the origin.

I decided to write a script to extract only the data I needed.  Essentially:

if (line.StartsWith("<gml:posList>"))
{
    outputFile.WriteLine(line);
}

This brought the file size from 1.9GB to 448MB, and the number of lines from 49,352,141 to 1,983,127.

With this smaller file, I could parse each data point to find the boundries.  This helped me determine the file size to use.  I used a ratio of 500 metres per pixel which, I felt, gave a good balance between detail and file size (although it is quite large, still).

Now that I had all the boundaries and all the points, I was able to plot them on an image using .Net’s graphics library.  A few dozen lines of code, and a few seconds of processing, and I had my map.

Code will be relased on my BitBucket shortly. Stay tuned!

Articles

HamOntFire – Visualizing @HFS_Incidents

In .Net, Hamilton on January 14, 2013 by Matt Grande Tagged: , , , , ,

Awhile back, I came across the @HFS_Incidents twitter account. It broadcasts all of the calls that Hamilton Fire Services responds to.

Being an Open Data guy, I was pretty happy to see this, but I thought Twitter wasn’t the best format. If I see an event, I might have no idea where it refers to. “0 Block GERTRUDE ST” isn’t helpful unless you already know where Gertrude Street is.  I also thought that this might be a good opportunity to get my hands dirty with Google Maps, Web Sockets, the Twitter API, and image generation.

The format of the tweets was pretty easy to parse, so I decided to throw together a few services and map the data out. And with that, HamOntFire was born.

At a high level, here’s what’s happening:

  • SuperWebSocket is constantly polling the Twitter API for new tweets
  • When new tweets are found…
    • I parse the data into an object (mostly to get the address into a Google-appropriate format)
    • I geocode the location using Google’s Geocoding API
    • I store the data in RavenDB
    • The event is pushed to the browser
  • The browser displays the tweets using Google Maps.

This was all fairly easy, with one exception: Both RavenDB and SuperWebSocket are dependent on Json.Net (Newtonsoft’s fantastic Json parser which has become the de facto standard), but each required very different versions (=4.0.8 and &ge;4.5.4, respectively). After some research, I discovered that this was easy to take care of in .Net, thanks to binding redirects:

</pre>
<dependentAssembly>
<assemblyIdentity name="Newtonsoft.Json" publicKeyToken="30ad4fe6b2a6aeed" culture="neutral" />
<bindingRedirect oldVersion="0.0.0.0-4.6.0.0" newVersion="4.0.8.0" />
 </dependentAssembly>
<pre>

This says “For any versions of Newtonsoft.Json between v0.0.0.0 and 4.6.0.0, just use version 4.0.8.0.”

Once that was taken care of, everything else was a piece of cake.

You can check out the site for yourself here, and check out the code on GitHub here.

Post Script: I’m also trying to come up with other statistics to display on the Stats page. If you can think of any, please let me know!

Articles

CoderCamp

In Uncategorized on November 23, 2012 by Matt Grande Tagged:

So I haven’t really talked much about CoderCamp.  I think it’s high time that I do.

What is CoderCamp?

CoderCamp is an unconference in the same spirit as BarCamp. We started organizing them in Hamilton in response to demand for a more skill-building and code-focused event for software developers to attend. StartupDrinks allows us to get to know one another andDemoCamp allows us show off what we’ve built and get feedback from the community, but we were missing an event that’s for coders and focused on software development.

CoderCamp is for developers to learn techniques and technologies from one another in a casual setting. We meet to talk about coding, to learn from each other, and get better at what we do in the process. You don’t have to give a talk to attend, but we welcome you to come, talk and share what you know as well.

Can you dumb it down a bit?

Nerds get together and watch short, usually about five minute, presentations about programming, technology, etc.  It’s a good time.

Sounds awesome.  When’s the next one?

As of this writing, December 6 at Chester’s Beers of The World, in Gore Park.  We usually meet at 6:30 and get started at 7.  If December 6 has come & gone, check out codercamp.org for more info.

I have something cool to present!

Cool!  Contact myself, or one of the other organizers.  If you don’t know any of us, just show up with your demo! There’s always room for more!

See you December 6!

Articles

Grande.Pluralizer is finally properly open source.

In Uncategorized on September 6, 2012 by Matt Grande

This post is out of date. I no longer use BitBucket. Grande.Pluralizer is now available on GitHub

I’ve finally put my Pluralization engine onto a DVCS.  This will make many people happy.

I’ve chosen to put it on BitBucket.  This will make many people unhappy.

I chose BB over GitHub for two reasons:

  1. BB seems better at Mercurial support (I prefer Hg to Git, but that’s another topic altogether)
  2. I prefer BB’s UI.  I seem to be in the minority on this one, but I find BB to be nicer to use.

So, if you like Grande.Pluralizer, please download it from BitBucket.  If you come across any errors, or things you would like to see, either issue a pull request or add it to the issue tracker.

Have fun!

https://bitbucket.org/mattgrande/grande.pluralizer

Follow

Get every new post delivered to your Inbox.