Big Data is not necessarily about crunching massive amounts of data – it’s about finding unrealized value in a data set. This might be done on huge sets of billions of records, using Map/Reduce running on thousands of servers. But it might also be done at a smaller scale by a concerned citizen.
More and more data is being made available by municipalities and countries. These are typical “big data” collections: Just a bunch of data of varying quality with very little associated metadata explaining what the data is about. And one professor in New York has shown what can be achieved with open data and careful analysis.
He discovered that just two fire hydrant generated $55,000 of parking tickets per year. Of course it is illegal to park in front of a fire hydrant, but a quick look at Google Streetview revealed that motorists were highly unlikely to notice these hydrants. After the professor’s analysis made it onto Reddit and then into the New York Post, the Department of Transportation changed the street marking to make it clear that these spaces were illegal. Consequently, New Yorkers are likely to save $55,000 a year (plus some frustration).
Let’s set more data free and see where it takes us.