New technology

SI Journalism unveiled at influential open source conference  

Next generation concepts in open-source technologies usher in a whole new style of journalism   

by Dan Crawford  

Online content is driving change in the world today. Grassroots activism groups have increasingly turned to the web as their preferred communication medium. The effectiveness of the online medium has helped bring many fringe and otherwise media-ignored issues out into the world for public discussion and open debate.

The issue of Peak Oil is one such example where the web has been instrumental in sweeping a relatively obscure topic out from under the carpet and into the pages of newspapers, the open airwaves of broadcast media, and even the stoic floors of political institutions.

The underlying foundation behind this type of awareness and activism is based on open-source software. Open-source software is code that can be freely used and modified. Development of the software is normally conducted by volunteers in a collaborative environment where the changes and improvements are openly shared. “Open source” sprouted from the technological community to combat proprietary software projects owned by corporations. In essence, open source supplies the digital world with freedom of expression.

The O'Reilly Open Source Convention, known more widely as OSCON, was recently held in Portland, Oregon, running from July 24 to 28. It is arguably the largest open-source event in the world, having a turnout of over 2,000 attendees—a significant number, considering the registration costs ranged from US$1,000 to $2,000 per person.

I covered the event with the hopes of unearthing the next technologies and tools that will be employed to drive future changes in our world. I was impressed by the number of interesting sessions and the revolutionary projects coming out of the open-source community. A relevant aside is that the impetus for much of this development can be traced back to the simple concept of the hypertext link, brainchild of Tim Berners-Lee, from which the worldwide web was born. This is a small but important point that exemplifies how one seemingly insignificant concept can poten-tially swing open the floodgates of human ingenuity and eventually alter the consciousness of society.

Standing out in front of the pack at OSCON was a presentation titled “Journalism via Program-ming,” given by Adrian Holovaty of The Washington Post. He began his talk by bluntly stating that “journalism is broken.” He then said that it’s broken not because of diminishing distribution trends, or the sinking stock prices of media companies, or the increases in media density, but rather because journalists just throw their used data out. The news organizations have the infra-structure to collect, edit and verify, and publish the infor-mation, but these very same organizations do not keep the information in a structured machine-readable format that would allow for recurring uses. This is in direct contrast, he noted, to online frameworks, such as Google Base, Wikipedia, and Craigslist, that are desperate for this type of information.

He went on to provide a number of examples where information is thrown out in newsrooms—daily collections of police crime reports, election results, restaurant lis-tings, local band listings, and classifieds are all regularly lost. The next half of his talk took the audience through what is possible if news corporations were to hold their data collections in structured formats. He provided real world examples using his own projects as a reference. One of his most well-known projects is chicagocrime.org, a site that inserts the daily Chicago police reports into a database, and then couples the data with Google maps. A web-accessible interface is then layered on top that allows the public to customize and refine map displays driven by a variety of user-selected criteria: crime type, street, date, police district, ZIP code, and ward, among others. A website that combines data and web services from more then one source is referred to as a “mashup.”

The other examples he cited are accessible from The Washington Post website: “Faces of the Fallen,” and the “Congress Votes Data-base.”

The “Faces of the Fallen” site keeps a running count of all US fatalities in Iraq and Afghanistan. The information is collected daily by Washington Post journalists and then updated to a database. A maps feature is tied in with the database to visually show how many have died from each state across the US. A user can zoom in on the map to individual states, cities, and even streets. Views can be selected displaying the infor-mation by age, year of death, home state, and military branch, as well as a user-specified ad-vanced search. Each fallen US service member is given his or her own page that provides a photo, hometown, age, time of death, unit, and details of the incident. The age view is most alarming, with its bar graph displaying deaths clearly showing the 20- to 22-year-olds suffering the most fatalities.

The “Congress Votes Database” automatically scrapes its info from various governmental websites on a daily basis. The data are then presented online in useful and understandable ways: most recent, absent voters, late-night votes, vote margins (narrow and wide), keynotes, and bills. Every congress member has a dedicated page where further information can be viewed, such as financial disclosure statements. A syndica-tion service (RSS) is also offered that can be user-customized to provide notification of vote results for a specific congressman or senator, which allows a user to easily keep watch over how an elected official is representing the electorate.

This new philosophy of journalism, being in its infancy, has not yet received an official name. Close cousins are CAR journalism (Computer-Assisted Reporting), and Analytical Journalism. Both of these, though, do not address the central problem of journalists being wasteful with their own infor-mation. The conceptual shift of adding value to all information collected through the use of structured formats, along with providing a means for the public to experience and understand the data through different views, could logically be termed “Structured Information Jour-nalism” or “SI Journalism.” If the name does stick, keep in mind that you read it here first in The Republic.

Other open-source projects worthy of mention are two privacy-related applications. A project coming out of Europe, called freenigma, is answering the call to ensure privacy of web-based email services. It’s easily installed as a web browser plug-in that adds encryption services to any web-based email account, like Google mail, Hotmail, and Yahoo. The other online privacy tool is called Tor. This software protects a user's network traffic from being tracked and monitored. This is an important issue, given the position of the Bush admin-istration on peoples’ rights to privacy. This tool is actively supported by the EFF (Electronic Frontier Foundation), an estab-lished privacy-activism group.

The mindset of the OSCON event was best captured by Canada’s IT and Security Trade Commissioner, Brian Vescovi. I asked him what stood out the most for him at the convention. He said it was “how focused on the technology these developers are, and how strident they are in their beliefs that they are not just developing software, but they are helping to change the world.”

For further information on the sessions and presentations at OSCON, visit the O’Reilly site at http://conferences.oreillynet.com/os2006/.