incommunicado

In anticipation of the forthcoming Alpha release of the new Kohana-based Swift River codebase on March 30th, Meedan is sponsoring a hack night and discussion on Friday, April 2, 2010 at 5pm, at the Meedan offices in San Francisco, California.

Proposed goals for the open-invite evening include:

• getting the new app running locally on some developer machines
• making sure that new developers are clear on how to use the git workflow
• discussing new features and the roadmap
• discussing integration potential, future points of collaboration & code sharing

We will probably transition to conversation over dinner.

WHERE?
There are two ways to participate in the hackup:

• In person. In San Francisco, at the new Meedan offices at 972 Mission St. See: http://www.inveneo.org/?q=mission_social
• Online. In the Swift River Public Skype channel

.

WHEN?
The alpha hackup is sandwiched between Where2.0 and Wherecamp — starting at 4/2 5pm PST and last until late, as the Ugandan and Kenyan teams come online (the morning of 4/3 Eastern Africa Time).

WHO SHOULD ATTEND?

• developers who are interested in getting started hacking on the new codebase
• developers with skills in PHP, MySQL, Python or Ruby on Rails
• our colleagues working on like-minded tools
• journalists or researchers who just want to learn more about the platform

RSVP
If you are planning on coming to the event in San Francisco just send an email to cgblow at gmail dot com — we will make sure you have a phone number so you can get in the Meedan office.

Hackathon Photo By SuperAmit

In a past life, before developing software, I was a musician. The two have a lot in common actually: recursive pattern, rhythm, syntax, meter. I suppose most developers don’t think of code this way, but I do. It needs to look as good to humans as it does to machines. When I took over development of Swiftriver and I was looking for a theme to weave through all of our releases, it was natural to default to what I love: music.

Each release of Swift River will carry the name of a style of African music. The release schedule appears below. I think it’s fitting to able to pay homage the music around me in this way and it actually serves as a starting point for people looking to be exposed new styles of music that they may not already know. The first version of Swift, an early Alpha, will be available on March 31st, 2010 and there will be regular updates and iterations to follow. If you’re interested in how Swift River verifies and filters the crowd, visit us here.

Alpha Releases
0.0.0 Rumba (Release Date: March 31st, 2010)
0.1.0 Apala
0.2.0 Batuque
0.3.0 Benga
0.4.0 Bikutsi
0.5.0 Cape Jazz
0.6.0 Chimurenga
0.7.0 Fuji
0.8.0 Harare
0.9.0 Jit

Beta Releases
1.0.0 Jùjú (August 1st, 2010)
1.1.0 Kizomba
1.2.0 Kuduro
1.3.0 Kwaito
1.4.0 Kwela
1.5.0 Makossa
1.6.0 Malouf
1.7.0 Maloya
1.8.0 Marrabenta
1.9.0 Museve
2.0.0 Mbalax

Non-Beta and Beyond
2.1.0 Mbaqanga
2.2.0 Mbube
2.3.0 Morna
2.4.0 Palm
2.5.0 Raï
2.6.0 Sakara
2.7.0 Sega
2.8.0 Soukous
2.9.0 Taarab
3.0.0 Zouk

Ouagadougou Drummer Photo by Babasteve

A guest post from the team running Ushahidi’s Situation Room at Columbia University’s School of International and Public Affairs (SIPA).

Congratulations to everyone involved with Ushahidi-Chile at SIPA for receiving a $10,000 grant from the Mozilla foundation! Our project aligned perfectly with Mozilla Foundation’s mission to promote openness, innovation and participation on the Internet. The work of crisis mapping is more than innovative as it directly affects the most vulnerable people at Chile who are struggling after the earthquake. Thanks to this grant, our volunteers will be able to train local Chileans on how to use and manage the platform themselves.

We hope to use a small part of this grant to recognize and thank our wonderful volunteers at SIPA. Although there was an explosion of interest throughout SIPA to help with this initiative, not everyone was able to take significant time out of midterm exams to help. Our core volunteers are motivated and compassionate; the more they help the more they keep coming back. We could not thank them enough for their work, and we are happy that this grant can go in recognizing the hundreds of hours they have put into Ushahidi-Chile. Thank you volunteers!

The majority of this grant will be instrumental in moving this initiative to the next phase: to Chile. We have been communicating with various Chilean organizations to transition the Ushahidi platform to local Chileans. What’s great about the Ushahidi platform is that it is open source and versatile. The immediate need for the platform was to report incidents on the map, but the need is shifting to another direction. What better people to think of how to best utilize this platform than Chileans themselves.

If you were there or following South by South West yesterday, you may have heard some chatter on Twitter about the Africa 3.0 talk by Teddy Ruge of Project Diaspora. In his panel he used Skype video to chat in real time with software developers and incubators in Cameroon, Kenya and with my staff in Uganda. Two of the developers from Appfrica, Moses Mugisha and Victor Miclovich appeared with me on camera to speak with the crowd. One of them, Victor, quickly discussed his natural language processing project SiLCC. Here’s a quick interview allowing him more time to explain his background, how he got into semantic programming and why peer learning is critical.

In the post Natural Language Processing with Swift River I introduced you to two underlying technologies powering Swiftriver. Victor Miclovich is the Ugandan volunteer developer who’s spent the last few months working to help make these plans reality with SiLCC (Swift Language Computation Core).

Victor Miclovich, SiLCC Developer and Volunteer

How did you get involved with natural language processing technologies? It’s not a field many Africans are known to be active in.

Victor Miclovich: When I got hooked up with writing code, I discovered another side of computing as a kid. That side of computing led me to doing heavy research work and this fired up my inquisitiveness.

NLP wasn’t what I played with first. I started with doing work inside of artificial intelligence which surprisingly had a likeliness to programming. As I matured in the area, I realized that one would never really master everything in A.I. (artificial intelligence) and so I narrowed my work to machine learning which was about 2 years ago.

Machine learning is a wide subject with lots of literature and research work being done in many areas from computer vision, speech recognition and natural language processing…the list is actually endless. I settled for computer vision work and NLP eventually because of their feasibility and ease of access to technology in Africa. I knew that getting a robot built could be a little bit hard! (laughs) That’s how I got involved with NLP technologies; my curiosity drove me to it.

What inspires you as a software developer?

VM: First, it is my drive and passion for technology. Being able to instruct a machine to do your bidding is something that brings a sense of fulfillment. People don’t always follow my instructions.

Secondly, the people (developers) I encounter wherever I work and go bring inspiration to me…this is just my way of saying that Appfricans are my inspiration…their accomplishments and determination is what keeps me going.

How do you see Africa’s role in tech changing over the next 20 years?

VM: Africa’s role in tech is slowly becoming visible. Universities in Africa are slowly churning up new grads every year. These grads have ambition and are tired of staying behind technology. This is what is going to drive the change in tech.

When students or people get tired of being behind, they develop a strong desire for change…we should not be pessimistic about this, we are optimistic! There are many floating examples all over Africa of tech communities and start-ups sprouting up.

You’re very involved in the community and helping the guys coming up behind you, giving gratis lectures and workshops at your university and mentoring your peers in your spare time. Why do you feel this is important?

VM: It is always important to give back to mankind. Philanthropy has it’s rewards. I feel that if I don’t do something, those are years lost to the community. I have lived in a place where I’ve seen folks with lots of potential and those that have made it in life and science (or tech). Many stay arrogant and don’t give back to the community…they end up living lavish lives with lots of wealth and of course, who else will suffer? The community will. It suffers because those well off folks only do things that will help themselves.

On the other side of things, my giving back to the community helps make more folks like me or even better than me. This means that we shall get thinkers rising exponentially and an increase of great ideas that won’t end up being recursively boring but wonderful! These are the main reasons I feel what I’m doing is important.

How has it been working with the global developer community? Have you learned a lot?

VM: Working with a global developer community has been very interesting. I’ve virtually met folks that have done cool stuff with their time and this has been quite inspiring. It has boosted the quality of the work that I do because of the huge amounts I learn from my peers in the global dev community.

You can follow Victor’s work on SiLCC here or on the Swift River mailing-list.

I’ve been told that crowdsourcing of elections isn’t a wise move. After all, what value will anyone gain from gathering a bunch of yammering “l33t-speak” texting reports from the unwashed masses? Election monitoring should only be done by trained volunteers and their results analyzed by professionals.

Last week I spent a couple days in sessions on election monitoring, with election monitoring professionals, some who have been doing this for many years. I’ll be the first to say that I didn’t know a lot about how they worked, so I listened very closely to understand just what was going on. I was surprised, very surprised, by what I heard.

As most people, I think of election monitors as those guys with a funny hat/shirt that stand to the side and watch and scribble things on a sheet of paper. That’s true, that is what they do.

However, I was surprised to learn that most of these election monitors are not well trained, give mediocre data, and some end up not just objective bystanders, but proactive enforcers of a given political party. In effect they aren’t much, if any, better than those aforementioned “unwashed masses”.

Exacerbating the issue is the fact that these election monitoring groups rarely keep good records of their monitors. Thus, when fraudulent or questionable activity comes in from a polling station, where compromised monitors are working, they aren’t tracked. In the next elections, these same people step forward and are then used again.

Finally, it doesn’t make sense to equate election monitoring with voting booth monitoring. What about everything that happens before and after elections? If a government is going to skew the elections, they’ll make all the preparations beforehand and don’t generally do much on election day; they wind up the clock and just let it tick they way they built it. By only monitoring voting stations one is basically making the case that what happens weeks/days/hours before the actual voting process is completely independent and has no influence on the elections.

Absurd. Elections are a process, not an event.

Interesting, and insane…

In the collection of data then (not the analyzing of it), why are election monitors so much more valuable than crowdsourcing from the public?

First off, there is something to be said about having someone sit at the polling station all day long. It could be argued that people just coming in to vote might see something, but someone there all day is much more likely to catch an inconsistency or fraud. (Though, it should be noted that most African voters defend their vote by also staying all day long until the local ballots are counted and announced.)

Second, election monitors are “known” and can therefore be found if a report was given that needs to be followed up on.

In a more sane and perfect world, we can see from these two points that election monitors would be useful. However, we don’t, and yet we keep seeing money get poured into this broken system. Acting as if election monitoring is some sacrosanct fortress, the only right way to make sure elections happen the right way.

A more sane person might suggest that you look at a lot of incoming data from many sources and use it all. Sure, keep some proven election monitoring practices in place, but balance that against the crowd who can provide you with alerts that might never have cropped up on your radar before. Marry that with budget tracking and cleaned census data.

I’m taking a sarcastic tone in this post, as it seems a bit beyond me that the crowdsourcing method gets pilloried when compared to election monitoring. Both are part of a greater ecosystem of data collection that should be happening. One isn’t better than the other overall, and both bring different strengths to the table. In effect, election monitoring is just pseudo-controlled crowdsourcing.

It’s time to start thinking of elections holistically, as a process and with many forms of data input and analysis.

Sean Conner recently asked a great question about integrating a Question and Answer service like Aardvark or Yahoo Answers into Swiftriver. Here is our approach at Team Swift…

In a Swift instance, Aardvark could be used as an additional ‘channel’ of input. Existing channels are Twitter, Email, SMS, News, RSS (any RSS feed), and Other (the catch-all for items coming in via our API). The only thing the Swift app wants to do is receive content, allow users and our algorithms to tag that content, and based on user behavior it scores the originating content source.

As an example for Aardvark: Johhny asks the question “Did an earthquake really happen in Chile?” on Vark.com on Feb 28th, only a day after the quake actually occurs. Robert responds on Vark with “No, at least I haven’t heard of one.” Vark user Jeremy responds with “Actually, Yes. An 8.8 magnitude earthquake occurred in Chile on Feb 27th.” In Swift, the answer and the accuracy of that answer is more important to us than the actual question (which just provides context).

To integrate Aardvark in Swift we’d probably write a module using their API that aggregates Answers with the corresponding Question as the ‘description’. Example of how that data would post to the Swiftriver API:

Title: “Actually, Yes. An 8.8 magnitude earthquake occurred in Chile on Feb 27th.”
Description: “Did an earthquake really happen in Chile? – Johnny”
Time: 17:08 EST
Date: Feb 30, 2010
Source: Jeremy’s user id on Vark.com
Channel: Vark API
Lat: 10.31
Lon: 01.40
Tags: 8.8, earthquake, chile

Title: “No, at least I haven’t heard of one.”
Description: “Did an earthquake really happen in Chile? – Johnny”
Time: 03:10 EST
Date: Feb 30, 2010
Source: Robert’s user id on Vark.com
Channel: Vark API
Lat: 10.31
Lon: 01.40
Tags: heard, chile, earthquake

Within Swift this is the primary information we need to verify information. Users with a careful eye will notice that we’ve included location data that Vark probably may or may not provide. We can easily extract that info from the hosted service SULSa. Here, the source is what we’re scoring. The channel is just an indicator for the user about where the content is coming from. That said, the source is not Vark itself, nor is it the user’s answer on Vark, but rather the user id on Vark.

Thus, if Robert keeps giving inaccurate answers, he maintains a very low score in Swift while Jeremy is viewed as the more trusted authority. Now this approach assumes that Vark.com offers an API that allows for this type of data aggregation which I don’t think they currently do. Perhaps, it’s a question for the Vark team?

Caroline Stauffer is a member of the core SIPA Team deploying the Ushahidi-Chile platform. She is a graduate student at Columbia University’s School of International and Public Affairs (SIPA) where she focuses on International Media and Communications. She spent the past summer working with the Associated Press in Bangkok, and worked for a nongovernmental organization in the Dominican Republic prior to SIPA.

Continuous aftershocks, office buildings scheduled for destruction, villages without access to water, gasoline shortages, and looting. These are some of the 800+ incidents my fellow students and I mapped during our first week in the situation room at Columbia University’s School of International and Public Affairs (SIPA). The death toll from the February 27 earthquake is much lower than the number of lives lost since January 12 in Haiti, but the 8.8-magnitude quake in southern Chile was one of the largest on record, and coordinating information, needs and responses on the ground is essential.

Near real-time crisis mapping

An initial training session at Columbia University the Monday after the earthquake drew 60 students—during SIPA’s midterm examination period. A core team of six students is working to ensure that the growing list of volunteer names translates into more hours spent monitoring and mapping. Other schools within Columbia University are starting to get involved, and the team has reached out to contacts at New York University. One of our core members took a quick break from the Situation Room to head to United Nations headquarters after individuals from the United Nations Development Program (UNDP) asked for training on the Ushahidi-Chile application. Most importantly, as students continue to monitor traditional and social media, more reports are reaching the Ushahidi platform from on the ground in Chile.

The core SIPA Team

Some students were initially skeptical of Ushahidi-Chile. The Chilean government had said international aid was not necessary, and the value of mapping information from news sources and Twitter—in short, information that can be found elsewhere—was questioned. However, by Monday, messages communicating the needs and locations of people who lacked basic supplies and water were coming in. It seemed that our team had uploaded enough information to be truly useful on the ground. Soon, Chilean organizations including Red Salvavidas and Chile Ayuda were contributing reports daily.

As a journalist, the flow of information within Ushahidi has been immensely interesting to me. The process that our teams of monitors, mappers and administrators go through to produce the Ushahidi page is not unlike the path a journalist takes to report and produce a story. In both cases, the reporter seeks information from overlooked sources and tries to verify the data. Small bits of information combine to produce a larger picture, giving an overview of a particular situation. In Ushahidi, the picture that emerges is an interactive map, rather than a narrated story.

Crisis Mapping volunteers at SIPA

The SIPA team has touched base with technology students from Talca, in one of the hardest regions hit in Chile. The Chilean students now have administrative abilities on the Ushahidi-Chile page. The goal of the SIPA students working with Chile-Ushahidi is to transfer the platform to an organization in Chile by the end of March, but to leave behind a trained group of crisis mappers at SIPA who will be ready to assemble and share information whenever and wherever the next disaster strikes.

One of the core features of Swift River is the Language Computation Core, or SiLCC as we like to call it (Swift Language Computation Component). Users send feeds to SiLCC which, using a number of machine learning techniques, parses the incoming text and extracts relevant keywords. The idea is that these keywords (tags) can then be used to infer taxonomic relationships between content items. Some camps refer to this as semantic programming, others refer to it as artificial intelligence, but the general concept remains the same: helping programs to perform tasks based on a growing series of complex conditions. In this case ‘auto-tagging’ or ‘predictive tagging’ based on conditions learned from user behavior and preset rules.

The diagrams below illustrate how this dataflow works. Text passing through SiLCC are parsed, tags are extracted, those tags are then reapplied in the Swift River UI. There, Swift attempts to build relationships between tags. (ex. items tagged with “chile” and “earthquake” are likely related. However items tagged ‘chili’ and ‘earthquake’ likely are not.) Of course other factors are considered like date, time, the point of origin and location of the content creator.

Swiftriver – SiLCC Dataflow

One of the services running within SiLCC is another service called SLISa, which we like to call Lisa (because the ’s’ is silent, hehe). SLISa is the Swift Language Improvement Service App and it trains SiLCC to learn from user interaction. When users of Swift edit or flag tags as inaccurate, SLISa is the service that creates all the conditions that helps SiLCC to learn from it’s mistakes and improve for the future.

Swiftriver – SLISa Dataflow

SiLCC is an open source project being developed in Python using the pyNLP toolkit. There’s several additional layers of text parsing that I haven’t touched upon including how SiLCC deals with SMS txtspk and Twitter picoformats like hashtags but more on that in a future post!

More on SiLCC at http://swift.ushahidi.com/extend/silcc/. If you have a passion for machine learning, large data sets, and intricate algorithms you might also consider joining the Swift River Google Group or our public Skype Chat.

The Alpha release of Swift River, Version 0.0.9 Rumba will be available to the public on March 31, 2010. Developers can find always find the latest working build and issue tracker at http://github.com/ushahidi/Swiftriver.

Mark Belinsky co-founded Digital Democracy with Emily Jacobi. He serves as Technical Director and brings a background in computer science, sociology and film & media studies. He and Emily have worked closely with Burmese populations since 2007.

Working in the tech sphere, it’s the power and passion that people have that never ceases to astound me. Following the Jan. 12 earthquake in Haiti, people around the world contributed an incredible outpouring of support for the people of Haiti. Now, as that support expands to Chile, it is evident that we are participating in a game changing moment.

On January 12th when the earthquake in Haiti struck, Digital Democracy had two team members on the ground looking at the economic livlihoods of young Haitians . Worried for their safety and the greater loss of life, my colleague Emily and I immediately joined Patrick in the Situation Room at Fletcher, and collaborated with the core team to establish a system that allowed hundreds of volunteers to glean emergency instances from the ground and place them on the map for response. Those first volunteers have trained many more, and their tireless work has directly saved lives.

When an 8.8 magnitude earthquake hit Chile on Saturday, we were called on again to train volunteers. The Fletcher Ushahidi team, their hands full with Haiti, was able to rapidly set up the site for Chile.Ushahidi.com, but needed help training new volunteers. Digital Democracy collaborated to adapt the training module for the Situation Room at the Fletcher School at Tufts University, and on Sunday night we conducted an hour+ training on Skype for a core team at SIPA at Columbia in New York City who are now running the Chile operations. Their ability to quickly respond to the devastating emergency in Chile, despite it being midterms week, speaks to the dedication and passion of the volunteers. The 10 people we trained on Sunday trained 40 more the next day at SIPA, plus a growing number of volunteers in Santiago.

For us, this has been a particularly rewarding process, as we’ve been following and working with Ushahidi since fall 2008. Watching Ushahidi evolve from mapping post-election violence in Kenya to elections reporting when we helped with VoteReportIndia, we’ve worked to harness Ushahidi to empower ordinary citizens. At Digital Democracy, we’ve applied Ushahidi to map human rights abuses in Burma through Handheld Human Rights and peace mapping in Kenya through Sisi Ni Amani – “We are Peace.” In each case, we’ve worked with local community organizations to determine needs and apply the technology – Ushahidi – to best meet their needs.

Our mission is to utilize technology for civic engagement, and these examples demonstrate how technology can encourage deeper engagement with the world around us. At the core, it’s not the technology but the people who use it. Ushahidi is a tool that’s a key part of our arsenal because of the open community around it. As technology enables more and more people to contribute to meaningful actions to save lives, I for one am excited to see where it leads.

This week in Nairobi has been “Election Monitoring” week due to the NDI/DFID meeting on Tue/Wed and the HIVOS meeting on Thur/Fri. Interestingly enough, both meetings heavily addressed the uses, or lack thereof, of technology in the election monitoring process.

One of the ideas that hit me was the need for a toolbox of technical tools that could be used by election monitoring groups and citizens both before, during and after the elections.

Understanding the Framework

Most of us think of an election as an event, I did too. Koki Muli provided us a with a great framework to understand the election process as a whole, using this visualization for everyone to see that it is indeed a long-term process, not an event.

The Election Cycle

Throughout the days of talk, a lot of information was given about elections, and specifically election monitoring as a practice and how it actually goes down in the real world. I’m not an expert in election monitoring, so there was a lot of education for me, but putting on my technology hat I (and the other tech guys present) were able to come up with some ideas for tactical-level solutions that might be useful to these election monitoring groups.

Starting with the Issues

I don’t have time to run through all of the issues, but here are a couple that have a technology component that could be created to help with them.

Legal Framework
The legal framework for elections starts many years before an election happens, but might be the most important element in an election (in a law-abiding state). These are high-brow activities, where law professors and legal experts write in a format that isn’t easily understood by ordinary people.

The question then is, how could this legalese be translated into a format that is digestable by normal citizens, and how could they then give feedback on what they think of these new laws? I’m already thinking of a tool that can be built for this, having a lot in common with (oddly enough) digital Bibles and how they allow multiple levels of commentary by readers and scholars.

Election Monitors
Registration of election monitors is not rocket science. Many times, the election monitoring groups get funds at the last minute and they go out and recruit anyone above a certain education level within the different constituencies. There is little vetting, some training, and no understanding of their values and ethics for when election day actually arrives. It’s no surprise then that when election day comes, many of these election monitors turn out to be little more than party infiltrators, sometimes abusing the system worse than normal citizens.

What tool can we build to support creation of a database of known election monitors, with full information about them, so that when irregularities or criminal activities are done by or with them that it can be tracked. Having a historical database of election monitors will allow for pattern mapping and even a blacklist for the civil society groups when they go out to find monitors for the next elections.

Real-time Data Collection
During the campaigning and the election day itself, it’s possible to collect information on irregularities from both trusted/known sources (including media, government and civil society), but also from ordinary citizens in the Ushahidi-style of crowdsourcing from citizenry. This includes media monitoring as well as gathering of historical and demographic data so that real-time analysis can happen.

Tools like Ushahidi and Managing News can be utilized here, this is what they are built for. The different types of news sources are complimentary, allowing layering of new, real-time data on top of demographic and historical election data.

Onto the Toolbox

These issues and challenges throughout the election cycle cannot be set up and deployed in the last few weeks or months if they are to be effective. We all need to sit down now, come up with the tactical and strategic-level tools that are needed and deploy them within the organizations who can best utilize them in the election process.

• What tools can be built?
• What tools exist already?
• How can we create a toolbox that election monitoring groups, and citizens, can choose tech tools from and deploy?
• Has this already been started somewhere else?

Add your thoughts and ideas in the comments below.

Let’s see if we can come up with some ideas, as technologists especially, of tools that can work in conjunction with one another to strengthen the election monitoring and the election cycle as a whole.

Reinier Battenberg had a good slideshow with examples of where tools are needed at different parts of the election cycle. I’m including that here as well as food for discussion. (click the image to download his Powerpoint presentation)

10 applications for use in election monitoring

Finally, here’s a list of election monitoring-specific tools we could think of, what would you add to it?

• Ushahidi
• FrontlineSMS
• Managing News
• OpenStreetMap
• SlingshotSMS
• RapidSMS
• SMS Reception Center
• Budget Tracking (app by SODNET)
• International SMS gateways (Clickatell, Intellisms, etc)
• Swift River
• SMS Turks
• Pentaho
• MySociety.org
• NDI’s election monitoring tool (ASP based)
• CiviCRM
• WordPress
• CMS tools (Drupal, Joomla, etc.)
• Freedom Fone

It is often very encouraging to see adoption and use of the Ushahidi platform in Africa, and the Togo Election map certainly adds to the excitement the Ushahidi team feels when individuals and organizations download, customize and spearhead projects.

The elections in Togo are being held today, and Kokou Etou set up this implementation to help aggregate information. He partnered with IFES (international Foundation for Electoral Systems) and other organizations working in Togo to visualize the info and put it online.

For some background on the Togo Elections, read this piece by Amourlaye Touréand Paul Chick of IFES/Togo.

There are three ways to submit information
1. Text your location and info to 0012673935789
2. Email info@togoelection2010.com
3. Submit reports on the site

Do support this initiative and visit their site here, please do let others know within Togo to check the site for information. This will be particularly helpful to Kokou and his partners as they have already done some preliminary outreach through online forums and radio ads.

Our dedicated team of volunteers have mapped over 100 reports including many pictures, and this less than 48 hours after the deployment of the Ushahidi-Chile platform. During this time, I worked directly with colleagues from Columbia University’s School of International and Public Affairs (SIPA), my alma mater, to help them set up their own Situation Room and take the lead on Ushahidi-Chile. Our partners Digital Democracy (D2) once again played an instrumental role and provided the SIPA Team with the full Ushahidi training they needed. Many thanks to both!

And of course thanks to The Fletcher Team for holding the fort during the first 48 hours. Last but certainly not least, thanks to the Ushahidi Tech Team for once again rallying to the cause after their massive effort on Haiti. Our colleague Oscar Salazar from “Ushahidi-Mexico” was also pivotal in getting the Ushahidi-Chile platform off the ground and will continue to be. Clearly, this is about building a community, a volunteer response community, more than anything else.

This is why I’m excited that another School of International Affairs has set up a Situation Room and taken the lead on an Ushahidi deployment. The Fletcher, DC, Geneva, London and Portland Teams already demonstrated the incredible multiplier effect that is possible thanks to their lead on the Ushahidi-Haiti project. There are now three core Ushahidi Situation Rooms in the world, Boston, New York and Geneva. They are fully trained and continue to train others. So if you are based at a university and want to set up your own Situation Room, then please feel free to contact me: patrick@ushahidi.com.

The earthquake in Chile and resulting tsunamis have not caused the widespread loss of life that many initially feared just hours after the 8.8 magnitude earthquake. But the advantage of platforms like Ushahidi’s is that they can be deployed regardless, just in case.  Consider this the Precautionary Principle of Disaster Management. And thanks to a growing community of CrisisMappers, these precautionary deployments can happen more often and faster.

Stay tuned for guest blog posts from the Digital Democracy and SIPA Teams.

I learned about the massive earthquake at 7:00 A.M. EST and immediately got in touch with the Ushahidi Tech Team to set up an Ushahidi-Chile platform. I then reached out to my colleagues from The Fletcher School at Tufts University and others who contributed their time to the Haiti deployment. They are are now responding to the earthquake in Chile and tsunami effected countries. This time, however, the volunteers are trained and the Ushahidi Tech Team simply cloned the Ushahidi-Haiti version for Chile. We’re already busy customizing the deployment for Chile. So everything is actually moving twice as fast and so is the CrisisMappers Group.

Customizing Ushahidi-Chile

We all realize that this is a risk coming so soon after the massive efforts around the Haiti quake. But we didn’t have all the answers then either. I will collaborate closely with the Fletcher/Tufts team and look for volunteer groups with universities in Mexico and Colombia, for example. I’ve already reached out to my good friend Oscar Salazar who launched the successful Ushahidi-Mexico platform and he is mobilizing his network of volunteers in Mexico. I’m also reaching out to our South American colleagues at GlobalVoices.

We’ve already set up a long code that individual can use to text in their location and urgent needs: +44 762.480.2524. Note that we are working with our partners to set up a short code for Chile. So the one listed here 4636 is not operational. Please standby for more information on a local short code. In the meantime, you can submit reports directly online here. You can also send an email to chile@ushahidi.com with specific location information and urgent needs.

We will provide further updates on this blog and also on our Twitter feed @ushahidi.

Jaroslav Valuch is our Ushahidi Field Representative. He was the first to join the Ushahidi-Haiti @ Tufts Team to deploy the platform and he has been in Haiti for over 3 weeks. Jaroslav is from the Czech Republic and a Hubert Humphrey/Fulbright Fellow at the Philip Merrill College of Journalism at the University of Maryland. In 2009 he worked in Burma as a relief and capacity building projects coordinator.

It’s been one month since I returned from holiday in the Czech Republic to Washington DC to carry on with my Hubert Humphrey Fellowship Program at University of Maryland, Philip Merrill College of Journalism. Yet in the plane I was thinking about where to start my professional affiliation, which is an essential part of my fellowship. Before Christmas, I had the chance to meet Patrick Meier from Ushahidi, chatting about my experience from Burma and about the potential of new technologies in human rights and development in general. He connected me to other people like the great folks from Digital Democracy.

I had no idea that few hours after my departure in DC in January I would be in Patrick’s living room at The Fletcher School in Boston tracking any available sources of information coming from the earthquake hit Haiti and submitting them to Ushahidi.
Things were moving incredibly quickly—more and more volunteers joining, more information available, especially after the 4636 short code was announced. In order to process the load of information, teams of developers were 24/7 working on the system, the team at Fletcher/Tufts kept improving the way of processing messages from hour to hour in order to ensure that they are processed as quickly as possible. In the mean time, trainings for dozens and hundreds of new volunteers were taking place together with online trainings for teams in Geneva, DC, Portland… The pace was incredibly rapid; it had its up and downs. It is a big responsibility to ask people in the most desperate situations you can imagine to send a text message to some anonymous number.

The initial plan was to simply map those incidents to give to anyone interested a clear picture of the situation on the ground. One simply cannot expect that the rescue teams and other responders will be online on Port-au-prince airport checking the Ushahidi web in real time. But luckily, as many times during the whole operation, amazing things happened: phone-call from US Coast Guards followed by requests to feed them with any incidents that are “actionable”—i.e., those that report accurate location and give specific information on the situation: trapped person, serious injuries, food and water shortage… The loop was closed. Things on the ground started moving.

However, that’s what we know now. At there time, there was no immediate feedback on what actions were taken based on the reports we provided. There was no time for that. We were just sitting in the Situation Room, watching the stream of messages and processing them, hoping that somebody on the ground was responding. Hundreds of volunteers did the same, putting all effort into something with no clear idea of it’s worth. We got pretty decent media coverage—it is sexy to report how new media are saving lives, but at that time nobody actually knew what exactly was going on. Not us in the center of it, and hardly anyone who was reporting it. We know now that some responders used Ushahidi-Haiti extensively.

I am in Port-au-Prince now ensuring that we will link the stream of reports from Ushahidi to coordination structures here on the ground. It is great to have unique reports from the ground accurately mapped, but without having a relevant responder on the other end it is only half of the job. The emergency phase is to large extent over and the early recovery phase requires more coordination and distribution of information to a variety of humanitarian actors. Closing this loop is priority again.

Single incidents reported through SMS are replaced by the needs of whole communities. The task is now to deal with the situation of hundreds of thousands people stacked in improvised IDP camps all around before the rainy season starts (and before they’ll get upset with the pace of humanitarian response). It will be difficult to provide everyone with proper shelter before the rains start. But for us, the question is simple: how can Ushahidi contribute to the overall effort to maximize the efficiency of the response?

Ushahidi was originally used to track individual violent incidents but is now transforming into a platform dealing with reports on the situation of populations, but still using the same source of information–short message sent by individuals. We obviously can’t expect that action will be taken on every single SMS report, unless it is a life/death situation such as serious injury. The power is in aggregation—more messages reporting similar incidents from one location mean more relevant and reliable report. Diversification of sources of information is needed, as well as improved verification of this information. The Haitian diaspora and other partners already stepped into this process and very soon the Ushahidi reporting will shift to a totally new dimension.

Ushahidi reporting won’t replace the needs assessments being conducted on the ground by teams of relief workers. But they can fill the gaps, indicate some irregularities, they can be sentinels of some emerging dangerous trends. Furthermore, local communities should have a channel that will bring their concerns to the right places. More and more we  hear complaints of numerous local initiatives that the whole humanitarian process is almost exclusively top-down. “We feel like foreigners in our own country” is only one of the many comments I have overheard. It is not easy for them to take part in high-level decision-making processes—lack of appropriate knowledge and experience in the UN driven coordination system might be only part of the problem, impossibility to pass security gates at UN logbase where most of the meetings take place is trivial but still very common obstacle.

And similarly, the people on the ground have to be informed about the humanitarian and development efforts. Internews and Thomson Reuters Foundation (TRF) took the lead in this—by providing support to local radio stations and journalists and by emergency information service sending essential information through SMS to the affected communities.

Deploying Ushahidi in Haiti was rapid with no preexisting structures and channels in place on the ground. And it is quite an exciting idea to imagine that next time the links will be already established. Simply, way less people will die.

The first two days in Port au Prince were quite surreal for me. Of course I knew the place already from the satellite imagery and Google Earth, but being here personally is a bit different. One gets quite used to the images of destruction around, especially with a lot of work to do and great supportive people around. But no matter how intense it was, the most impressive experience for me are still the first days at Fletcher school with all the great people around; the energy, commitment and readiness to accept responsibility is something I will hardly forget.

Denise Roz Sewell is on the core team of Ushahidi-Haiti @ Tufts and has been instrumental in connecting with our partners at Crisis Camp. Roz is a Pickering Fellow and was a Fulbright scholar in Morocco prior to joining Fletcher. She shares a startling experience with us here.

Capacity. Comparative Advantage. Workflow. Crisis Camp. Ticket Tracking.

Before working with Ushahidi Haiti @ Tufts University (UH@T), all of the above words rarely, if ever, entered into my vocabulary. Well, to be fair, I did say comparative advantage a lot, but that was just because of an economics course I was in. The others, however, were not things I would talk about or even knew about. However, after Crisis Camp Boston came to Fletcher on January 23, I happily altered my vocabulary.

CrisisCamp @ Fletcher. Source: Carol Waters

This all actually begins on January 18, the first day UH@T began processing and geo-locating text messages arriving from the Haitian short-code 4636. At first, a small team of people could accomplish this task. We received about 100-150 messages a day, and our backlog was only 400 messages. (The short-code had started two days before we had put this team together.) However, the 4636 Haitian media campaign drastically changed all of that. We started receiving over 1,000 messages a day, and the four-person SMS team could no longer handle it.

We had reached our capacity.

UH@T’s comparative advantage is our ability to create a visualization of need with ridiculously accurate GPS coordinates; however, for the SMS team, we were getting stuck on the back-end. Currently, the way the Ushahidi platform is set-up, you need to have administrative access to report SMS messages coming from an outside system. I completely understand that feature. First, the messages contain personal information. Second, only a handful of messages (consistently around 30%) are actionable. This means a message that an aid organization can take action on, i.e. requests for food/water/shelter/medicine. Therefore, we needed to shift through 1000 messages searching for actionable items with a four-person team.

Ushahidi Back End

Obviously, this is not a productive workflow.

In came the savior of the UH@T operation: Crisis Camp. They had heard of UH@T and wanted to help. This team of developers was ready and willing to work on any project we wanted, and obviously the SMS workflow, or lack there of, became a priority. So I sat down with Ashirul, our Crisis Camp liaison, to develop a better system. A few users sort these messages deciding which are actionable; then, the UH@T team of volunteers geo-locates them, and then the UH@T core team maps them directly to the Ushahidi website. That way, the 300 or so actionable messages we received in a day could be mapped quickly by the 300 or so volunteers we were training without worrying about the stress of creating 300 or so new admin logins for the Haiti-Ushahidi platform. Then, the core team, the new experts on geo-locating, could double-check the coordinates before everything went live on the website. After Ashirul left, I laughed to myself. This was an ideal workflow, and to be honest, I thought it was a pipe dream.

This was when the phrase “ticket tracking system” became my new favorite.

SMS Management System

RT (Request Tracket) is a web platform that takes items, called ‘tickets,’ and manages them for the users within the system. It is fully customizable, allowing you to specify which users see which tickets and how tickets flow through the system from user to user. Using version three of this system and making source-level modification, the Crisis Camp volunteers started to make the ideal workflow a reality. Ian, pictured below, understood the importance of seeing this through to the end, so that at 1 am, after Crisis Camp had ended, he was down in the SitRoom with the rest of us still plugging away. He was a fast addition to the UH@T Tech Team, and by January 30, RT and our new workflow went live.

Now there have been bugs in the system, the server has crashed more than once, and Ian has received more than one (ok more than 10) crazy emails from me at 1 am. Through it all, though, we have been able to do our job because of this new platform. Let me give you an example, using our old Ushahidi and four-person team system working around the clock, we geo-located and reported about 30 – 40 messages in a day. The first full day of using RT, we put 150 new reports on haiti.ushahidi.com.

Personally, I will change my vocabulary for that kind of improvement any day.

Cross-posted on Patrick Meier’s blog iRevolution.

What if we could communicate with disaster affected communities in real-time just days after a major disaster like the quake in Haiti? That is exactly what happened thanks to a partnership between the Emergency Information Service (EIS), InSTEDD, Ushahidi, Haitian Telcos and the US State Department. Just 4 days after the earthquake, Haitians could text their location and urgent needs to “4636” for free.

I will focus primarily on the way that Ushahidi used 4636. Since the majority of incoming text messages were in Creole, we needed a translation service. My colleague Brian Herbert from Ushahidi and Robert Munro of Energy for Opportunity thus built a dedicated interface for crowdsourcing this step and reached out to dozens of Haitian communities groups to aid in the translation, categorization and geo-location of every message, quickly mobilizing 100s of motivated and dedicated volunteers. So not only was Ushahidi crowdsourcing crisis information in near real-time but also crowdsourcing translation in near real-time.So not only was Ushahidi crowdsourcing crisis information in near real-time but also crowdsourcing translation in near real-time.

Text messages are translated into English just minutes after they leave a mobile phone in Haiti. The translated messages then appear directly on the Ushahidi platform. The screenshots below (click on graphics to enlarge) illustrates how the process works. The original SMS in Creole (or French) is displayed in the header. In order to view the translation, one simply clicks on “Read More”.

Ushahidi Back End

-

Incoming Text Messages

-

View Translated Text

If further information is required, then one can reply to the sender of the text message directly from the Ushahidi platform. This is an important feature for several reasons. First, this allows for two-way communication with disaster affected communities. Second, an important number of messages we received were not actionable because of insufficient location information. The reply feature allowed us to get more precise information.

The screenshots below show how the “Send Reply” feature works. We weren’t sure if Universite Wayal was the same as Royal University. So we replied and asked for more location information. Note the preset replies in both English and Creole. The presets include thanks & requests for more location information, for example. Of course, one is not limited to these presets. Any text can be typed in and sent back to the sender of the original SMS. This feature has been part of the Ushahidi for almost two years now. We send off the request for more information and receive the following reply within minutes.

Two-Way Communication

-

Preset Replies

When we receive an urgent and actionable SMS like this one, we can immediately create a report. By actionable, we mean there is sufficient location information and the description of the need is specific enough to respond to, just like the example above.

Creating a Report

First, the GPS coordinates for the location is identified. This can be done directly from the Ushahidi platform by entering the street address or town name. Sometimes a bit of detective work is needed to pinpoint the exact coordinates. Next, a title and description for the report is included–the latter usually comprising the text of the SMS. This is what we mean by structured information. The report is then tagged based on the category framework. Pictures can be uploaded with the report, and links to videos can also be included. Finally the report is saved and then approved for publication.

This is how the Ushahidi-Haiti @ Tufts team mapped 1,500+ text messages on the Ushahidi platform. We are now working with Samasource and Crowdflower to have the translation work serve as a source of income for Haitians inside Haiti. But how does all this connect to response?

Ushahidi’s “Get Alerts” feature is one of my favorite because it allows responders themselves to customize the specific type of actionable information that is important to them; i.e., demand driven situational awareness in near real-time. Not only can responders elect to receive automated alerts via email, but they can also do so via SMS. Responders can also specify their geographic area of interest.

Subscribe to Alerts

For example, if a relief worker from the Red Cross has a field office in neighborhood of Delmas, they can subscribe to Ushahidi to receive information on all reports originating from their immediate vicinity by specifying a radius, as shown below.

Selecting Area of Interest

The above Alerts feature is now being upgraded to the one depicted below, which was designed by my colleague Caleb Bell from Ushahidi. Not only are responders able to specify their geographic area of interest, but they can also select the type of alert (e.g., collapsed building, food shortage, looting, etc.) they want to receive. They can even add key words of interest to them, such as “water”, “violence” or “UN”. The goal is to provide responders with an unprecedented degree of customization to ensure they receive exactly the kind of alerts that they can respond to.

Highly Customized Alerts

On a more “macro” level, I recently reached out to colleagues at the EC’s Joint Research Center (JRC) to leverage their automated sentiment (”mood”) analysis platform. Sentiment Analysis is a branch of natural language processing (NLP) that seeks to quantify positive vs negative perceptions; akin to “tone” analysis. I suggested that we use their platform on the incoming text messages from Haiti to get a general sense of changing mood on an hourly basis. I’ll blog about the results shortly. In the meantime, here’s a previous blog post on the use of Sentiment Analysis for early warning.

Patrick Philippe Meier

The outstanding volunteer team at Tufts University has played an instrumental role in Ushahidi’s response to Haiti. They  trained hundreds of volunteers and set up Situation Rooms in Washington DC, Geneva, London and Portland. Together, they mapped over 3,000 urgent and actionable reports on Ushahidi-Haiti from a multitude of sources including text messages from Haiti. They also coordinated directly with responders on the ground and helped save hundreds of lives.

We often get the question: why Tufts? The short answer is that I happen to be completing my PhD at The Fletcher School. So when the earthquake struck Haiti, I was able to reach out to friends and colleagues on campus for their help. But little did I know just how much of an incredible movement was about to be launched. Our fellow volunteers at Tufts and at other Situation Rooms around the world have consistently gone above and beyond.

This is why Ushahidi is now formally partnering with Ushahidi-Haiti @ Tufts who will take the lead on Ushahidi-Haiti from here-on. I will of course remain directly connected to the team as Ushahidi’s Advisor to Ushahidi-Haiti @ Tufts. In addition, we have a full-time Ushahidi Field Representative in Haiti, my colleague Jaroslav Valuch, to directly support the team at Tufts University. The Situation Rooms in Geneva and London will also remain active.

The purpose of the Tufts initiative is to work closely with the Haitian Diaspora, local civil society groups and the Government of Haiti. Why? Because the latter have expressed a strong interest in using Ushahidi-Haiti to help them inform the post-disaster reconstruction and development process. They want to give Haitians the ability to hold the development community accountable for the way their country is rebuilt. The Tufts team will work directly with and for these stakeholders so that Ushahidi-Haiti can become a fully Haitian owned and managed project.

This model is one that we at Ushahidi are very interested in replicating in the future. The largest concentration of Ushahidi-trained volunteers in the world are now in Boston. They have the ability to lead future deployments and train volunteers in many other universities to do the same. They are also taking the lead in partnering with a Diaspora, and this too could be an ideal model for future deployments. In short, the team at Tufts is continuing to break new ground for Ushahidi every day.

Ushahidi Tip of the Day: Follow the Ushahidi-Haiti @ Tufts team closely as they take the field of Crisis Mapping to a whole new level.

In continuing its commitment to the development of Ushahidi’s work in Kenya, Ushahidi has just completed a six-month pilot project with the Kenya AIDS NGO’s Consortium (KANCO) to  visualize civil society organizations (CSOs) in Kenya providing responses to the HIV/AIDS and tuberculosis (TB) epidemics. The project, in collaboration with AIDSPortal, UK, involved modifying the Ushahidi platform for Crowdsourcing Crisis Information in order for KANCO to map its member organizations. This work contributes to KANCO’s mission, “to provide leadership, promote collaboration and enhance capacity among CSOs and other stakeholders to respond to HIV & AIDS and TB at the community level.”

KANCO provides leadership in community systems strengthening (CSS) and policy and advocacy activities across the country. KANCO has extensive experience leading workshops and training for organizations in the field of HIV/AIDS and TB, and these standards for providing CSS were utilized to develop a workshop curriculum, based on the customized Ushahidi platform. In November, two workshops were conducted with 16 participants to gather feedback on platform’s usability. This feedback informed the final customization and last week, Feb. 9-11, a further 23 participants were trained at the KANCO offices in Nairobi. The workshops introduced the three partner organizations, provided background on the project, website demonstration, and user training. Participants were given an hour and a half to browse the website. In particular, they were instructed to search for organizations on the map, learn who’s doing what, where in Kenya, submit their organization’s profile and subscribe to the alerts feature on the map.

User Testing Guidelines

Overall, the 3 days of training were a great success. Participants responded positively to the project and enjoyed using the site, especially adding and updating their profiles. Generally participants thought the site was a good tool for networking and partnering, knowledge building, promotion and marketing, advocacy and information sharing. One participant described her experience as, “I have confidence and now I can do it for myself because I have been delegating rather than doing it myself. Others can find us online and may be interested in working with us.”

Workshop Participants adding Organization Profiles

Currently the KANCO instance does not have a mobile component, but participants liked idea of adding SMS to the site. They thought SMS would be beneficial to them, especially for keeping up-to-date with KANCO and their constituents, particularly in regards to referrals. Participants commented that people are more likely to check SMS than email on a regular basis and that people are comfortable communicating via SMS.

Established in 1990, KANCO is a networking organization with a cumulative membership of over 1000 CSOs. The partnership between KANCO, Ushahidi, and AIDSPortal was established in early 2009 and the pilot project was supported by a Geochallenge Grant, awarded to the team in August 2009. For updates on the project, keep an eye on the KANCO blog.

Post compiled with Jamie Lundine, Health and Social Geography Consultant/KANCO Map Project Manager. Follow her on Twitter @jlundine.