flyingblogspot.com (tales from urban dilettantia)

It was watching Rammstein’s lead singer, Till Lindemann, ride a giant penis cannon across the stage that got me thinking.  I’d spent the day wandering around the Perth show of Australia’s biggest music festival, the Big Day Out, and as I watched Lindemann frolic on the cannon, spraying the joyful crowd with foam, it occurred to me that I’d seen an awful lot of cock.*

Now I’m a reasonably well-read woman and I’ve spent enough of my time around activists and feminist historians to know the deal. Structural oppression making it tough for women to ascend the slippery ladder of rock music fame, complex economic and historical issues, blah blah blah. I appreciate that there’s a wider context, and sure, it takes a little more effort to find and book bands with female musicians, and I understand tour promoters are in this game to make money. I get this; I really do.**

But was it possible that, during a whole day at the country’s biggest festival, there hadn’t been one female musician on either of the main stages?  As a curious data analyst I decided to find out more, and pulled together a spreadsheet with every act and every individual musician performing at the Perth 2011 Big Day Out.  You may draw your own conclusions:

Five minutes on the Googletubes indicates that The New Pornographers, Rasputina, The Firey Furnaces, Portishead, Shonen Knife, Skunk Anansie, Amanda Palmer, Florence and the Machine, The Postal Service, Goldfrapp, The Breeders, Le Tigre, Belle & Sebastian, God Speed You! Black Emperor,Juliana Hatfield, P.J. Harvey, Morcheeba, Okkervil River, Regina Spektor, Silversun Pickups, Architecture in Helsinki, Broken Social Scene, The Magnetic Fields, of Montreal, Tender Trap, Tegan and Sara, Jebediah, Moriarty, New Rules For Boats, Schvendes, Cat Power, The Jezabels, Sneaky Sound System, Sparkadia, The Arcade Fire, Paramore, Ladyhawke, Tori Amos, Lacuna Coil and Beth Orton weren’t home to take the Big Day Out promoters’ call.

(See, I can do half-arsed research and find bands with women in them.  I’m sure the people who are paid to do this stuff could do the same. It’s not that hard.)

* That said, anyone walking into a Die Antwoord show should expect cock.

** I also get that we’re not likely to see a sharp increase of women in rock, punk or metal until we promoters book more female role-models on the big festival stages, but that deserves its own rant.

In the spirit of giving you fair warning, if you’re not into Australian Rules Football or into data analysis, move along before you taint your eyes with the horrible mash-up of the two that follows.

Now, fair warning given, anyone who has had the pleasure of me herding them into an inescapable corner and ranting at them about standard deviation will know that I enjoy playing AFL Dream Team during the football season.    There’s nothing quite like hanging over the barrier at a game to yell ‘Oi, ya lazy #^%#!  Kick it, don’t handball it!” at one’s star recruit.  (Particularly if you’re also yelling ‘TACKLE HIM!!1!’ at your other star recruit who is on the opposing team.)  But most of all, I enjoy it because it’s fundamentally a game of statistics, and there are few things I love so hard as I love stats.

And so, I have a bit of a summer project going on this year.  The thing with Dream Team is that there are a bunch of players that everyone will have because they’re obviously going to (a) rise in value or (b) be consistent.  These players can be picked out quite readily by skimming the media or the plethora of Dream Team blogs and other resources that have come into being over the past few years.  The two things that differentiate a great Dream Team player from a middle of the road one are trading strategy and picking up relatively cheap players who unexpectedly come good.

The trading strategy is something I messed up a little this season just gone and will be working on, but my off-season project is all about the latter – trying to determine whether there are any early indicators of players who are about to have a good season.  As a first step, I’ve gone through a bunch of data I’ve managed to scrape from the web and hacked together a bit of an Excel model to help me pick out a pool of players to study.  (It turns out – not unexpectedly – that there are a lot of players who have a respectable second season after a low-averaging start as a rookie, but very few players who exhibit a dramatic jump in form from middle-of-the-road to Dream Team gun in years two to five.  In fact, far less than one would believe, given all the blog and forum chatter around the elusive ‘breakout year’).

Having identified these players, I’m going to look in more detail at their averages, games played, consistency and so forth in the year immediately preceding their ‘breakout year’ to see whether they share any common characteristics not observable in non-breakout players.  As a sideline, I’m also going to look at the second-year players who have demonstrated a significant improvement from their rookie form, although I think the reasons for this (and the likely players) tend to be a bit more obvious to begin with.  Here’s a screencap of the work-in-process with a bit more detail around the proposed  methodology:

Yes, this is truly what I do for fun on my lunch-break.  I reckon it beats shopping for shoes by a factor of about eleventy million.

Happy Friday! I am declaring today to be particularly good, even as Happy Fridays go, since (1) I went for a run this morning, (2) the cubefarm has exploded into the amusing chaos of Yet Another Desk Reshuffle and (3) the first person I saw as I walked towards my building this morning was oliverm, smiling and waving at me. Good things.

There will be a Day 2 of Happiness Posting – in fact I will post it tonight, because tonight I miraculously have a night to myself which I intend to use for blogging and painting and the like. (I’ll also take this moment to point out that I never used the words ‘consecutive days’. And in the words of Nick Hornby, yes, that is a sneaky lawyer’s trick.)

In the meantime, have a little of my pontification on one of the Many Shiny Things I am excited about. This particular shiny thing is data. And datasets. Datamining. Visualisation. Information. I accept that this is a field that is dead sexy only to a very specific sub-set of people. However – trust me on this, unbelievers – for those of us wired in that particular way, it can be an intricate, exquisite, fascinating thing.

The web is beginning to engage with data in increasingly interesting ways. For one thing, free datasets are becoming more and more accessible and people are using them in ways that are sometimes artistic, sometimes functional, and very often both. While the plague of inaccessible data, siloed in institutions and organisations, still represents an incredible waste of potential, the situation is certainly improving. And, from an entirely different direction, Web2.0 technology has delivered the tools to easily collect one’s own raw data.

On the latter point, I’ve been running a small personal data collection project recently. Applications such as MapMyRide, FourSquare, Last.fm, LibraryThing, Sleep Cycles for iPhone and Delicious track a whole lot of stats already in a fairly passive, low-effort manner. In addition to those, I’ve adopted Your Flowing Data (YFD) to aggregate information on a number of other variables. There’s a YFD iPhone app, and a spiffy hack for Latitude users. (The very pretty Daytum tool also provides similar functionality.)

In a move that most consider an odd choice, I’ve made most of my staggeringly banal YFD data public, on a page called Banalytics (yes, I’m proud of that one). The reasons I’ve decided to open it up are various, but I’m particularly interested in the way it massively reduces my tendency to tell small, pointless lies, and feels like a gesture to towards understanding that people will choose to like me or not like me just as I am. (And of course there are the cynical days when I wonder whether maintaining privacy for the sake of privacy is a drain on my resources, and no more than a shared delusion.)

On a less personal and more academic level, I’m utterly fascinated by people who create large scale projects of this kind. Nicholas Felton is one of the best-known examples, and his personal Annual Reports have received plenty of coverage. (He also posts some really lovely stuff over at Tumblr!) His passion for design, information, for the appreciation of the very small – these things resonate with me and I can lose myself for great lengths of time in the existential detail of his work.

I struggle to find the right words to explain why I find this field so enchanting; it is a discipline of numbers and forms, not well suited to words. The attraction for me has much to do with shapes and patterns and relationships. Both the analysis and the visualisation are acts of beauty; acts of untangling immense webs, and of deft slicing and assembly. They are acts of perceiving the interconnectedness of things, and acts of holding that up and saying ‘see what I have found; see that it has meaning’. And they are the great heart – each heartbeat counted and illustrated – of the the intersection between the analytical and the designed.

For anyone interested in reading further, see below for a rambling assortment of the data blogs, tools, resources and datasets currently available on the web.

Data & Visualisation Blogs:
Data Wrangling
Flowing Data
DataBlog (The Guardian)
Information is Beautiful
Infosthetics

Datasets
Australian Bureau of Statistics
Data.gov (US)
UK Data Archive
UN Data
WHO Data and Statistics
OECD.Stat Extracts
Numbrary
Infochimps
DBPedia
UCI Machine Learning Repository
Time Series Data Library

Meta-lists of Datasets:
DataWrangling List
Datasets for Data Mining

Techniques:
Statistical Data Mining Tutorials