incommunicado

Sean Conner recently asked a great question about integrating a Question and Answer service like Aardvark or Yahoo Answers into Swiftriver. Here is our approach at Team Swift…

In a Swift instance, Aardvark could be used as an additional ‘channel’ of input. Existing channels are Twitter, Email, SMS, News, RSS (any RSS feed), and Other (the catch-all for items coming in via our API). The only thing the Swift app wants to do is receive content, allow users and our algorithms to tag that content, and based on user behavior it scores the originating content source.

As an example for Aardvark: Johhny asks the question “Did an earthquake really happen in Chile?” on Vark.com on Feb 28th, only a day after the quake actually occurs. Robert responds on Vark with “No, at least I haven’t heard of one.” Vark user Jeremy responds with “Actually, Yes. An 8.8 magnitude earthquake occurred in Chile on Feb 27th.” In Swift, the answer and the accuracy of that answer is more important to us than the actual question (which just provides context).

To integrate Aardvark in Swift we’d probably write a module using their API that aggregates Answers with the corresponding Question as the ‘description’. Example of how that data would post to the Swiftriver API:

Title: “Actually, Yes. An 8.8 magnitude earthquake occurred in Chile on Feb 27th.”
Description: “Did an earthquake really happen in Chile? – Johnny”
Time: 17:08 EST
Date: Feb 30, 2010
Source: Jeremy’s user id on Vark.com
Channel: Vark API
Lat: 10.31
Lon: 01.40
Tags: 8.8, earthquake, chile

Title: “No, at least I haven’t heard of one.”
Description: “Did an earthquake really happen in Chile? – Johnny”
Time: 03:10 EST
Date: Feb 30, 2010
Source: Robert’s user id on Vark.com
Channel: Vark API
Lat: 10.31
Lon: 01.40
Tags: heard, chile, earthquake

Within Swift this is the primary information we need to verify information. Users with a careful eye will notice that we’ve included location data that Vark probably may or may not provide. We can easily extract that info from the hosted service SULSa. Here, the source is what we’re scoring. The channel is just an indicator for the user about where the content is coming from. That said, the source is not Vark itself, nor is it the user’s answer on Vark, but rather the user id on Vark.

Thus, if Robert keeps giving inaccurate answers, he maintains a very low score in Swift while Jeremy is viewed as the more trusted authority. Now this approach assumes that Vark.com offers an API that allows for this type of data aggregation which I don’t think they currently do. Perhaps, it’s a question for the Vark team?

Caroline Stauffer is a member of the core SIPA Team deploying the Ushahidi-Chile platform. She is a graduate student at Columbia University’s School of International and Public Affairs (SIPA) where she focuses on International Media and Communications. She spent the past summer working with the Associated Press in Bangkok, and worked for a nongovernmental organization in the Dominican Republic prior to SIPA.

Continuous aftershocks, office buildings scheduled for destruction, villages without access to water, gasoline shortages, and looting. These are some of the 800+ incidents my fellow students and I mapped during our first week in the situation room at Columbia University’s School of International and Public Affairs (SIPA). The death toll from the February 27 earthquake is much lower than the number of lives lost since January 12 in Haiti, but the 8.8-magnitude quake in southern Chile was one of the largest on record, and coordinating information, needs and responses on the ground is essential.

Near real-time crisis mapping

An initial training session at Columbia University the Monday after the earthquake drew 60 students—during SIPA’s midterm examination period. A core team of six students is working to ensure that the growing list of volunteer names translates into more hours spent monitoring and mapping. Other schools within Columbia University are starting to get involved, and the team has reached out to contacts at New York University. One of our core members took a quick break from the Situation Room to head to United Nations headquarters after individuals from the United Nations Development Program (UNDP) asked for training on the Ushahidi-Chile application. Most importantly, as students continue to monitor traditional and social media, more reports are reaching the Ushahidi platform from on the ground in Chile.

The core SIPA Team

Some students were initially skeptical of Ushahidi-Chile. The Chilean government had said international aid was not necessary, and the value of mapping information from news sources and Twitter—in short, information that can be found elsewhere—was questioned. However, by Monday, messages communicating the needs and locations of people who lacked basic supplies and water were coming in. It seemed that our team had uploaded enough information to be truly useful on the ground. Soon, Chilean organizations including Red Salvavidas and Chile Ayuda were contributing reports daily.

As a journalist, the flow of information within Ushahidi has been immensely interesting to me. The process that our teams of monitors, mappers and administrators go through to produce the Ushahidi page is not unlike the path a journalist takes to report and produce a story. In both cases, the reporter seeks information from overlooked sources and tries to verify the data. Small bits of information combine to produce a larger picture, giving an overview of a particular situation. In Ushahidi, the picture that emerges is an interactive map, rather than a narrated story.

Crisis Mapping volunteers at SIPA

The SIPA team has touched base with technology students from Talca, in one of the hardest regions hit in Chile. The Chilean students now have administrative abilities on the Ushahidi-Chile page. The goal of the SIPA students working with Chile-Ushahidi is to transfer the platform to an organization in Chile by the end of March, but to leave behind a trained group of crisis mappers at SIPA who will be ready to assemble and share information whenever and wherever the next disaster strikes.

One of the core features of Swift River is the Language Computation Core, or SiLCC as we like to call it (Swift Language Computation Component). Users send feeds to SiLCC which, using a number of machine learning techniques, parses the incoming text and extracts relevant keywords. The idea is that these keywords (tags) can then be used to infer taxonomic relationships between content items. Some camps refer to this as semantic programming, others refer to it as artificial intelligence, but the general concept remains the same: helping programs to perform tasks based on a growing series of complex conditions. In this case ‘auto-tagging’ or ‘predictive tagging’ based on conditions learned from user behavior and preset rules.

The diagrams below illustrate how this dataflow works. Text passing through SiLCC are parsed, tags are extracted, those tags are then reapplied in the Swift River UI. There, Swift attempts to build relationships between tags. (ex. items tagged with “chile” and “earthquake” are likely related. However items tagged ‘chili’ and ‘earthquake’ likely are not.) Of course other factors are considered like date, time, the point of origin and location of the content creator.

Swiftriver – SiLCC Dataflow

One of the services running within SiLCC is another service called SLISa, which we like to call Lisa (because the ’s’ is silent, hehe). SLISa is the Swift Language Improvement Service App and it trains SiLCC to learn from user interaction. When users of Swift edit or flag tags as inaccurate, SLISa is the service that creates all the conditions that helps SiLCC to learn from it’s mistakes and improve for the future.

Swiftriver – SLISa Dataflow

SiLCC is an open source project being developed in Python using the pyNLP toolkit. There’s several additional layers of text parsing that I haven’t touched upon including how SiLCC deals with SMS txtspk and Twitter picoformats like hashtags but more on that in a future post!

More on SiLCC at http://swift.ushahidi.com/extend/silcc/. If you have a passion for machine learning, large data sets, and intricate algorithms you might also consider joining the Swift River Google Group or our public Skype Chat.

The Alpha release of Swift River, Version 0.0.9 Rumba will be available to the public on March 31, 2010. Developers can find always find the latest working build and issue tracker at http://github.com/ushahidi/Swiftriver.

Mark Belinsky co-founded Digital Democracy with Emily Jacobi. He serves as Technical Director and brings a background in computer science, sociology and film & media studies. He and Emily have worked closely with Burmese populations since 2007.

Working in the tech sphere, it’s the power and passion that people have that never ceases to astound me. Following the Jan. 12 earthquake in Haiti, people around the world contributed an incredible outpouring of support for the people of Haiti. Now, as that support expands to Chile, it is evident that we are participating in a game changing moment.

On January 12th when the earthquake in Haiti struck, Digital Democracy had two team members on the ground looking at the economic livlihoods of young Haitians . Worried for their safety and the greater loss of life, my colleague Emily and I immediately joined Patrick in the Situation Room at Fletcher, and collaborated with the core team to establish a system that allowed hundreds of volunteers to glean emergency instances from the ground and place them on the map for response. Those first volunteers have trained many more, and their tireless work has directly saved lives.

When an 8.8 magnitude earthquake hit Chile on Saturday, we were called on again to train volunteers. The Fletcher Ushahidi team, their hands full with Haiti, was able to rapidly set up the site for Chile.Ushahidi.com, but needed help training new volunteers. Digital Democracy collaborated to adapt the training module for the Situation Room at the Fletcher School at Tufts University, and on Sunday night we conducted an hour+ training on Skype for a core team at SIPA at Columbia in New York City who are now running the Chile operations. Their ability to quickly respond to the devastating emergency in Chile, despite it being midterms week, speaks to the dedication and passion of the volunteers. The 10 people we trained on Sunday trained 40 more the next day at SIPA, plus a growing number of volunteers in Santiago.

For us, this has been a particularly rewarding process, as we’ve been following and working with Ushahidi since fall 2008. Watching Ushahidi evolve from mapping post-election violence in Kenya to elections reporting when we helped with VoteReportIndia, we’ve worked to harness Ushahidi to empower ordinary citizens. At Digital Democracy, we’ve applied Ushahidi to map human rights abuses in Burma through Handheld Human Rights and peace mapping in Kenya through Sisi Ni Amani – “We are Peace.” In each case, we’ve worked with local community organizations to determine needs and apply the technology – Ushahidi – to best meet their needs.

Our mission is to utilize technology for civic engagement, and these examples demonstrate how technology can encourage deeper engagement with the world around us. At the core, it’s not the technology but the people who use it. Ushahidi is a tool that’s a key part of our arsenal because of the open community around it. As technology enables more and more people to contribute to meaningful actions to save lives, I for one am excited to see where it leads.

This week in Nairobi has been “Election Monitoring” week due to the NDI/DFID meeting on Tue/Wed and the HIVOS meeting on Thur/Fri. Interestingly enough, both meetings heavily addressed the uses, or lack thereof, of technology in the election monitoring process.

One of the ideas that hit me was the need for a toolbox of technical tools that could be used by election monitoring groups and citizens both before, during and after the elections.

Understanding the Framework

Most of us think of an election as an event, I did too. Koki Muli provided us a with a great framework to understand the election process as a whole, using this visualization for everyone to see that it is indeed a long-term process, not an event.

The Election Cycle

Throughout the days of talk, a lot of information was given about elections, and specifically election monitoring as a practice and how it actually goes down in the real world. I’m not an expert in election monitoring, so there was a lot of education for me, but putting on my technology hat I (and the other tech guys present) were able to come up with some ideas for tactical-level solutions that might be useful to these election monitoring groups.

Starting with the Issues

I don’t have time to run through all of the issues, but here are a couple that have a technology component that could be created to help with them.

Legal Framework
The legal framework for elections starts many years before an election happens, but might be the most important element in an election (in a law-abiding state). These are high-brow activities, where law professors and legal experts write in a format that isn’t easily understood by ordinary people.

The question then is, how could this legalese be translated into a format that is digestable by normal citizens, and how could they then give feedback on what they think of these new laws? I’m already thinking of a tool that can be built for this, having a lot in common with (oddly enough) digital Bibles and how they allow multiple levels of commentary by readers and scholars.

Election Monitors
Registration of election monitors is not rocket science. Many times, the election monitoring groups get funds at the last minute and they go out and recruit anyone above a certain education level within the different constituencies. There is little vetting, some training, and no understanding of their values and ethics for when election day actually arrives. It’s no surprise then that when election day comes, many of these election monitors turn out to be little more than party infiltrators, sometimes abusing the system worse than normal citizens.

What tool can we build to support creation of a database of known election monitors, with full information about them, so that when irregularities or criminal activities are done by or with them that it can be tracked. Having a historical database of election monitors will allow for pattern mapping and even a blacklist for the civil society groups when they go out to find monitors for the next elections.

Real-time Data Collection
During the campaigning and the election day itself, it’s possible to collect information on irregularities from both trusted/known sources (including media, government and civil society), but also from ordinary citizens in the Ushahidi-style of crowdsourcing from citizenry. This includes media monitoring as well as gathering of historical and demographic data so that real-time analysis can happen.

Tools like Ushahidi and Managing News can be utilized here, this is what they are built for. The different types of news sources are complimentary, allowing layering of new, real-time data on top of demographic and historical election data.

Onto the Toolbox

These issues and challenges throughout the election cycle cannot be set up and deployed in the last few weeks or months if they are to be effective. We all need to sit down now, come up with the tactical and strategic-level tools that are needed and deploy them within the organizations who can best utilize them in the election process.

• What tools can be built?
• What tools exist already?
• How can we create a toolbox that election monitoring groups, and citizens, can choose tech tools from and deploy?
• Has this already been started somewhere else?

Add your thoughts and ideas in the comments below.

Let’s see if we can come up with some ideas, as technologists especially, of tools that can work in conjunction with one another to strengthen the election monitoring and the election cycle as a whole.

Reinier Battenberg had a good slideshow with examples of where tools are needed at different parts of the election cycle. I’m including that here as well as food for discussion. (click the image to download his Powerpoint presentation)

10 applications for use in election monitoring

Finally, here’s a list of election monitoring-specific tools we could think of, what would you add to it?

• Ushahidi
• FrontlineSMS
• Managing News
• OpenStreetMap
• SlingshotSMS
• RapidSMS
• SMS Reception Center
• Budget Tracking (app by SODNET)
• International SMS gateways (Clickatell, Intellisms, etc)
• Swift River
• SMS Turks
• Pentaho
• MySociety.org
• NDI’s election monitoring tool (ASP based)
• CiviCRM
• WordPress
• CMS tools (Drupal, Joomla, etc.)
• Freedom Fone

It is often very encouraging to see adoption and use of the Ushahidi platform in Africa, and the Togo Election map certainly adds to the excitement the Ushahidi team feels when individuals and organizations download, customize and spearhead projects.

The elections in Togo are being held today, and Kokou Etou set up this implementation to help aggregate information. He partnered with IFES (international Foundation for Electoral Systems) and other organizations working in Togo to visualize the info and put it online.

For some background on the Togo Elections, read this piece by Amourlaye Touréand Paul Chick of IFES/Togo.

There are three ways to submit information
1. Text your location and info to 0012673935789
2. Email info@togoelection2010.com
3. Submit reports on the site

Do support this initiative and visit their site here, please do let others know within Togo to check the site for information. This will be particularly helpful to Kokou and his partners as they have already done some preliminary outreach through online forums and radio ads.

Our dedicated team of volunteers have mapped over 100 reports including many pictures, and this less than 48 hours after the deployment of the Ushahidi-Chile platform. During this time, I worked directly with colleagues from Columbia University’s School of International and Public Affairs (SIPA), my alma mater, to help them set up their own Situation Room and take the lead on Ushahidi-Chile. Our partners Digital Democracy (D2) once again played an instrumental role and provided the SIPA Team with the full Ushahidi training they needed. Many thanks to both!

And of course thanks to The Fletcher Team for holding the fort during the first 48 hours. Last but certainly not least, thanks to the Ushahidi Tech Team for once again rallying to the cause after their massive effort on Haiti. Our colleague Oscar Salazar from “Ushahidi-Mexico” was also pivotal in getting the Ushahidi-Chile platform off the ground and will continue to be. Clearly, this is about building a community, a volunteer response community, more than anything else.

This is why I’m excited that another School of International Affairs has set up a Situation Room and taken the lead on an Ushahidi deployment. The Fletcher, DC, Geneva, London and Portland Teams already demonstrated the incredible multiplier effect that is possible thanks to their lead on the Ushahidi-Haiti project. There are now three core Ushahidi Situation Rooms in the world, Boston, New York and Geneva. They are fully trained and continue to train others. So if you are based at a university and want to set up your own Situation Room, then please feel free to contact me: patrick@ushahidi.com.

The earthquake in Chile and resulting tsunamis have not caused the widespread loss of life that many initially feared just hours after the 8.8 magnitude earthquake. But the advantage of platforms like Ushahidi’s is that they can be deployed regardless, just in case.  Consider this the Precautionary Principle of Disaster Management. And thanks to a growing community of CrisisMappers, these precautionary deployments can happen more often and faster.

Stay tuned for guest blog posts from the Digital Democracy and SIPA Teams.

I learned about the massive earthquake at 7:00 A.M. EST and immediately got in touch with the Ushahidi Tech Team to set up an Ushahidi-Chile platform. I then reached out to my colleagues from The Fletcher School at Tufts University and others who contributed their time to the Haiti deployment. They are are now responding to the earthquake in Chile and tsunami effected countries. This time, however, the volunteers are trained and the Ushahidi Tech Team simply cloned the Ushahidi-Haiti version for Chile. We’re already busy customizing the deployment for Chile. So everything is actually moving twice as fast and so is the CrisisMappers Group.

Customizing Ushahidi-Chile

We all realize that this is a risk coming so soon after the massive efforts around the Haiti quake. But we didn’t have all the answers then either. I will collaborate closely with the Fletcher/Tufts team and look for volunteer groups with universities in Mexico and Colombia, for example. I’ve already reached out to my good friend Oscar Salazar who launched the successful Ushahidi-Mexico platform and he is mobilizing his network of volunteers in Mexico. I’m also reaching out to our South American colleagues at GlobalVoices.

We’ve already set up a long code that individual can use to text in their location and urgent needs: +44 762.480.2524. Note that we are working with our partners to set up a short code for Chile. So the one listed here 4636 is not operational. Please standby for more information on a local short code. In the meantime, you can submit reports directly online here. You can also send an email to chile@ushahidi.com with specific location information and urgent needs.

We will provide further updates on this blog and also on our Twitter feed @ushahidi.

Jaroslav Valuch is our Ushahidi Field Representative. He was the first to join the Ushahidi-Haiti @ Tufts Team to deploy the platform and he has been in Haiti for over 3 weeks. Jaroslav is from the Czech Republic and a Hubert Humphrey/Fulbright Fellow at the Philip Merrill College of Journalism at the University of Maryland. In 2009 he worked in Burma as a relief and capacity building projects coordinator.

It’s been one month since I returned from holiday in the Czech Republic to Washington DC to carry on with my Hubert Humphrey Fellowship Program at University of Maryland, Philip Merrill College of Journalism. Yet in the plane I was thinking about where to start my professional affiliation, which is an essential part of my fellowship. Before Christmas, I had the chance to meet Patrick Meier from Ushahidi, chatting about my experience from Burma and about the potential of new technologies in human rights and development in general. He connected me to other people like the great folks from Digital Democracy.

I had no idea that few hours after my departure in DC in January I would be in Patrick’s living room at The Fletcher School in Boston tracking any available sources of information coming from the earthquake hit Haiti and submitting them to Ushahidi.
Things were moving incredibly quickly—more and more volunteers joining, more information available, especially after the 4636 short code was announced. In order to process the load of information, teams of developers were 24/7 working on the system, the team at Fletcher/Tufts kept improving the way of processing messages from hour to hour in order to ensure that they are processed as quickly as possible. In the mean time, trainings for dozens and hundreds of new volunteers were taking place together with online trainings for teams in Geneva, DC, Portland… The pace was incredibly rapid; it had its up and downs. It is a big responsibility to ask people in the most desperate situations you can imagine to send a text message to some anonymous number.

The initial plan was to simply map those incidents to give to anyone interested a clear picture of the situation on the ground. One simply cannot expect that the rescue teams and other responders will be online on Port-au-prince airport checking the Ushahidi web in real time. But luckily, as many times during the whole operation, amazing things happened: phone-call from US Coast Guards followed by requests to feed them with any incidents that are “actionable”—i.e., those that report accurate location and give specific information on the situation: trapped person, serious injuries, food and water shortage… The loop was closed. Things on the ground started moving.

However, that’s what we know now. At there time, there was no immediate feedback on what actions were taken based on the reports we provided. There was no time for that. We were just sitting in the Situation Room, watching the stream of messages and processing them, hoping that somebody on the ground was responding. Hundreds of volunteers did the same, putting all effort into something with no clear idea of it’s worth. We got pretty decent media coverage—it is sexy to report how new media are saving lives, but at that time nobody actually knew what exactly was going on. Not us in the center of it, and hardly anyone who was reporting it. We know now that some responders used Ushahidi-Haiti extensively.

I am in Port-au-Prince now ensuring that we will link the stream of reports from Ushahidi to coordination structures here on the ground. It is great to have unique reports from the ground accurately mapped, but without having a relevant responder on the other end it is only half of the job. The emergency phase is to large extent over and the early recovery phase requires more coordination and distribution of information to a variety of humanitarian actors. Closing this loop is priority again.

Single incidents reported through SMS are replaced by the needs of whole communities. The task is now to deal with the situation of hundreds of thousands people stacked in improvised IDP camps all around before the rainy season starts (and before they’ll get upset with the pace of humanitarian response). It will be difficult to provide everyone with proper shelter before the rains start. But for us, the question is simple: how can Ushahidi contribute to the overall effort to maximize the efficiency of the response?

Ushahidi was originally used to track individual violent incidents but is now transforming into a platform dealing with reports on the situation of populations, but still using the same source of information–short message sent by individuals. We obviously can’t expect that action will be taken on every single SMS report, unless it is a life/death situation such as serious injury. The power is in aggregation—more messages reporting similar incidents from one location mean more relevant and reliable report. Diversification of sources of information is needed, as well as improved verification of this information. The Haitian diaspora and other partners already stepped into this process and very soon the Ushahidi reporting will shift to a totally new dimension.

Ushahidi reporting won’t replace the needs assessments being conducted on the ground by teams of relief workers. But they can fill the gaps, indicate some irregularities, they can be sentinels of some emerging dangerous trends. Furthermore, local communities should have a channel that will bring their concerns to the right places. More and more we  hear complaints of numerous local initiatives that the whole humanitarian process is almost exclusively top-down. “We feel like foreigners in our own country” is only one of the many comments I have overheard. It is not easy for them to take part in high-level decision-making processes—lack of appropriate knowledge and experience in the UN driven coordination system might be only part of the problem, impossibility to pass security gates at UN logbase where most of the meetings take place is trivial but still very common obstacle.

And similarly, the people on the ground have to be informed about the humanitarian and development efforts. Internews and Thomson Reuters Foundation (TRF) took the lead in this—by providing support to local radio stations and journalists and by emergency information service sending essential information through SMS to the affected communities.

Deploying Ushahidi in Haiti was rapid with no preexisting structures and channels in place on the ground. And it is quite an exciting idea to imagine that next time the links will be already established. Simply, way less people will die.

The first two days in Port au Prince were quite surreal for me. Of course I knew the place already from the satellite imagery and Google Earth, but being here personally is a bit different. One gets quite used to the images of destruction around, especially with a lot of work to do and great supportive people around. But no matter how intense it was, the most impressive experience for me are still the first days at Fletcher school with all the great people around; the energy, commitment and readiness to accept responsibility is something I will hardly forget.

Denise Roz Sewell is on the core team of Ushahidi-Haiti @ Tufts and has been instrumental in connecting with our partners at Crisis Camp. Roz is a Pickering Fellow and was a Fulbright scholar in Morocco prior to joining Fletcher. She shares a startling experience with us here.

Capacity. Comparative Advantage. Workflow. Crisis Camp. Ticket Tracking.

Before working with Ushahidi Haiti @ Tufts University (UH@T), all of the above words rarely, if ever, entered into my vocabulary. Well, to be fair, I did say comparative advantage a lot, but that was just because of an economics course I was in. The others, however, were not things I would talk about or even knew about. However, after Crisis Camp Boston came to Fletcher on January 23, I happily altered my vocabulary.

CrisisCamp @ Fletcher. Source: Carol Waters

This all actually begins on January 18, the first day UH@T began processing and geo-locating text messages arriving from the Haitian short-code 4636. At first, a small team of people could accomplish this task. We received about 100-150 messages a day, and our backlog was only 400 messages. (The short-code had started two days before we had put this team together.) However, the 4636 Haitian media campaign drastically changed all of that. We started receiving over 1,000 messages a day, and the four-person SMS team could no longer handle it.

We had reached our capacity.

UH@T’s comparative advantage is our ability to create a visualization of need with ridiculously accurate GPS coordinates; however, for the SMS team, we were getting stuck on the back-end. Currently, the way the Ushahidi platform is set-up, you need to have administrative access to report SMS messages coming from an outside system. I completely understand that feature. First, the messages contain personal information. Second, only a handful of messages (consistently around 30%) are actionable. This means a message that an aid organization can take action on, i.e. requests for food/water/shelter/medicine. Therefore, we needed to shift through 1000 messages searching for actionable items with a four-person team.

Ushahidi Back End

Obviously, this is not a productive workflow.

In came the savior of the UH@T operation: Crisis Camp. They had heard of UH@T and wanted to help. This team of developers was ready and willing to work on any project we wanted, and obviously the SMS workflow, or lack there of, became a priority. So I sat down with Ashirul, our Crisis Camp liaison, to develop a better system. A few users sort these messages deciding which are actionable; then, the UH@T team of volunteers geo-locates them, and then the UH@T core team maps them directly to the Ushahidi website. That way, the 300 or so actionable messages we received in a day could be mapped quickly by the 300 or so volunteers we were training without worrying about the stress of creating 300 or so new admin logins for the Haiti-Ushahidi platform. Then, the core team, the new experts on geo-locating, could double-check the coordinates before everything went live on the website. After Ashirul left, I laughed to myself. This was an ideal workflow, and to be honest, I thought it was a pipe dream.

This was when the phrase “ticket tracking system” became my new favorite.

SMS Management System

RT (Request Tracket) is a web platform that takes items, called ‘tickets,’ and manages them for the users within the system. It is fully customizable, allowing you to specify which users see which tickets and how tickets flow through the system from user to user. Using version three of this system and making source-level modification, the Crisis Camp volunteers started to make the ideal workflow a reality. Ian, pictured below, understood the importance of seeing this through to the end, so that at 1 am, after Crisis Camp had ended, he was down in the SitRoom with the rest of us still plugging away. He was a fast addition to the UH@T Tech Team, and by January 30, RT and our new workflow went live.

Now there have been bugs in the system, the server has crashed more than once, and Ian has received more than one (ok more than 10) crazy emails from me at 1 am. Through it all, though, we have been able to do our job because of this new platform. Let me give you an example, using our old Ushahidi and four-person team system working around the clock, we geo-located and reported about 30 – 40 messages in a day. The first full day of using RT, we put 150 new reports on haiti.ushahidi.com.

Personally, I will change my vocabulary for that kind of improvement any day.

Cross-posted on Patrick Meier’s blog iRevolution.

What if we could communicate with disaster affected communities in real-time just days after a major disaster like the quake in Haiti? That is exactly what happened thanks to a partnership between the Emergency Information Service (EIS), InSTEDD, Ushahidi, Haitian Telcos and the US State Department. Just 4 days after the earthquake, Haitians could text their location and urgent needs to “4636” for free.

I will focus primarily on the way that Ushahidi used 4636. Since the majority of incoming text messages were in Creole, we needed a translation service. My colleague Brian Herbert from Ushahidi and Robert Munro of Energy for Opportunity thus built a dedicated interface for crowdsourcing this step and reached out to dozens of Haitian communities groups to aid in the translation, categorization and geo-location of every message, quickly mobilizing 100s of motivated and dedicated volunteers. So not only was Ushahidi crowdsourcing crisis information in near real-time but also crowdsourcing translation in near real-time.So not only was Ushahidi crowdsourcing crisis information in near real-time but also crowdsourcing translation in near real-time.

Text messages are translated into English just minutes after they leave a mobile phone in Haiti. The translated messages then appear directly on the Ushahidi platform. The screenshots below (click on graphics to enlarge) illustrates how the process works. The original SMS in Creole (or French) is displayed in the header. In order to view the translation, one simply clicks on “Read More”.

Ushahidi Back End

-

Incoming Text Messages

-

View Translated Text

If further information is required, then one can reply to the sender of the text message directly from the Ushahidi platform. This is an important feature for several reasons. First, this allows for two-way communication with disaster affected communities. Second, an important number of messages we received were not actionable because of insufficient location information. The reply feature allowed us to get more precise information.

The screenshots below show how the “Send Reply” feature works. We weren’t sure if Universite Wayal was the same as Royal University. So we replied and asked for more location information. Note the preset replies in both English and Creole. The presets include thanks & requests for more location information, for example. Of course, one is not limited to these presets. Any text can be typed in and sent back to the sender of the original SMS. This feature has been part of the Ushahidi for almost two years now. We send off the request for more information and receive the following reply within minutes.

Two-Way Communication

-

Preset Replies

When we receive an urgent and actionable SMS like this one, we can immediately create a report. By actionable, we mean there is sufficient location information and the description of the need is specific enough to respond to, just like the example above.

Creating a Report

First, the GPS coordinates for the location is identified. This can be done directly from the Ushahidi platform by entering the street address or town name. Sometimes a bit of detective work is needed to pinpoint the exact coordinates. Next, a title and description for the report is included–the latter usually comprising the text of the SMS. This is what we mean by structured information. The report is then tagged based on the category framework. Pictures can be uploaded with the report, and links to videos can also be included. Finally the report is saved and then approved for publication.

This is how the Ushahidi-Haiti @ Tufts team mapped 1,500+ text messages on the Ushahidi platform. We are now working with Samasource and Crowdflower to have the translation work serve as a source of income for Haitians inside Haiti. But how does all this connect to response?

Ushahidi’s “Get Alerts” feature is one of my favorite because it allows responders themselves to customize the specific type of actionable information that is important to them; i.e., demand driven situational awareness in near real-time. Not only can responders elect to receive automated alerts via email, but they can also do so via SMS. Responders can also specify their geographic area of interest.

Subscribe to Alerts

For example, if a relief worker from the Red Cross has a field office in neighborhood of Delmas, they can subscribe to Ushahidi to receive information on all reports originating from their immediate vicinity by specifying a radius, as shown below.

Selecting Area of Interest

The above Alerts feature is now being upgraded to the one depicted below, which was designed by my colleague Caleb Bell from Ushahidi. Not only are responders able to specify their geographic area of interest, but they can also select the type of alert (e.g., collapsed building, food shortage, looting, etc.) they want to receive. They can even add key words of interest to them, such as “water”, “violence” or “UN”. The goal is to provide responders with an unprecedented degree of customization to ensure they receive exactly the kind of alerts that they can respond to.

Highly Customized Alerts

On a more “macro” level, I recently reached out to colleagues at the EC’s Joint Research Center (JRC) to leverage their automated sentiment (”mood”) analysis platform. Sentiment Analysis is a branch of natural language processing (NLP) that seeks to quantify positive vs negative perceptions; akin to “tone” analysis. I suggested that we use their platform on the incoming text messages from Haiti to get a general sense of changing mood on an hourly basis. I’ll blog about the results shortly. In the meantime, here’s a previous blog post on the use of Sentiment Analysis for early warning.

Patrick Philippe Meier

The outstanding volunteer team at Tufts University has played an instrumental role in Ushahidi’s response to Haiti. They  trained hundreds of volunteers and set up Situation Rooms in Washington DC, Geneva, London and Portland. Together, they mapped over 3,000 urgent and actionable reports on Ushahidi-Haiti from a multitude of sources including text messages from Haiti. They also coordinated directly with responders on the ground and helped save hundreds of lives.

We often get the question: why Tufts? The short answer is that I happen to be completing my PhD at The Fletcher School. So when the earthquake struck Haiti, I was able to reach out to friends and colleagues on campus for their help. But little did I know just how much of an incredible movement was about to be launched. Our fellow volunteers at Tufts and at other Situation Rooms around the world have consistently gone above and beyond.

This is why Ushahidi is now formally partnering with Ushahidi-Haiti @ Tufts who will take the lead on Ushahidi-Haiti from here-on. I will of course remain directly connected to the team as Ushahidi’s Advisor to Ushahidi-Haiti @ Tufts. In addition, we have a full-time Ushahidi Field Representative in Haiti, my colleague Jaroslav Valuch, to directly support the team at Tufts University. The Situation Rooms in Geneva and London will also remain active.

The purpose of the Tufts initiative is to work closely with the Haitian Diaspora, local civil society groups and the Government of Haiti. Why? Because the latter have expressed a strong interest in using Ushahidi-Haiti to help them inform the post-disaster reconstruction and development process. They want to give Haitians the ability to hold the development community accountable for the way their country is rebuilt. The Tufts team will work directly with and for these stakeholders so that Ushahidi-Haiti can become a fully Haitian owned and managed project.

This model is one that we at Ushahidi are very interested in replicating in the future. The largest concentration of Ushahidi-trained volunteers in the world are now in Boston. They have the ability to lead future deployments and train volunteers in many other universities to do the same. They are also taking the lead in partnering with a Diaspora, and this too could be an ideal model for future deployments. In short, the team at Tufts is continuing to break new ground for Ushahidi every day.

Ushahidi Tip of the Day: Follow the Ushahidi-Haiti @ Tufts team closely as they take the field of Crisis Mapping to a whole new level.

In continuing its commitment to the development of Ushahidi’s work in Kenya, Ushahidi has just completed a six-month pilot project with the Kenya AIDS NGO’s Consortium (KANCO) to  visualize civil society organizations (CSOs) in Kenya providing responses to the HIV/AIDS and tuberculosis (TB) epidemics. The project, in collaboration with AIDSPortal, UK, involved modifying the Ushahidi platform for Crowdsourcing Crisis Information in order for KANCO to map its member organizations. This work contributes to KANCO’s mission, “to provide leadership, promote collaboration and enhance capacity among CSOs and other stakeholders to respond to HIV & AIDS and TB at the community level.”

KANCO provides leadership in community systems strengthening (CSS) and policy and advocacy activities across the country. KANCO has extensive experience leading workshops and training for organizations in the field of HIV/AIDS and TB, and these standards for providing CSS were utilized to develop a workshop curriculum, based on the customized Ushahidi platform. In November, two workshops were conducted with 16 participants to gather feedback on platform’s usability. This feedback informed the final customization and last week, Feb. 9-11, a further 23 participants were trained at the KANCO offices in Nairobi. The workshops introduced the three partner organizations, provided background on the project, website demonstration, and user training. Participants were given an hour and a half to browse the website. In particular, they were instructed to search for organizations on the map, learn who’s doing what, where in Kenya, submit their organization’s profile and subscribe to the alerts feature on the map.

User Testing Guidelines

Overall, the 3 days of training were a great success. Participants responded positively to the project and enjoyed using the site, especially adding and updating their profiles. Generally participants thought the site was a good tool for networking and partnering, knowledge building, promotion and marketing, advocacy and information sharing. One participant described her experience as, “I have confidence and now I can do it for myself because I have been delegating rather than doing it myself. Others can find us online and may be interested in working with us.”

Workshop Participants adding Organization Profiles

Currently the KANCO instance does not have a mobile component, but participants liked idea of adding SMS to the site. They thought SMS would be beneficial to them, especially for keeping up-to-date with KANCO and their constituents, particularly in regards to referrals. Participants commented that people are more likely to check SMS than email on a regular basis and that people are comfortable communicating via SMS.

Established in 1990, KANCO is a networking organization with a cumulative membership of over 1000 CSOs. The partnership between KANCO, Ushahidi, and AIDSPortal was established in early 2009 and the pilot project was supported by a Geochallenge Grant, awarded to the team in August 2009. For updates on the project, keep an eye on the KANCO blog.

Post compiled with Jamie Lundine, Health and Social Geography Consultant/KANCO Map Project Manager. Follow her on Twitter @jlundine.

Today it’s our pleasure to announce support from the Hivos Foundation, a Dutch non-governmental organization that aims to contribute to a fair, free and sustainable world. Hivos confirmed their support to Ushahidi several weeks ago, but we’ve been waiting for the perfect opportunity to put their funds to use. The recent deployment in Haiti presents just such an occasion.

This relationship with Hivos is very new for us, but it’s hard to understate how much impact their contribution has made in such a short amount of time.

Documenting the Haiti Deployment

In the immediate aftermath of the disaster, Ushahidi’s talented team of programmers and designers launched http://haiti.ushahidi.com. Next we focused on getting information into the system, and volunteer teams of crisis mappers gathered in situation rooms in Boston, DC and Geneva to “crowdsource the filter” and make sense of the mountain of incoming data.

Working in partnership with the U.S. Department of State, InSTEDD and others, we also launched the 4636 short code which has made mobile reporting in Haiti viable. Subscribers on the DigiCel and Voila / Comcel networks can send SMS free of charge and report their location and need. Together these two networks represent 89% coverage of the mobile market in Haiti. And we’ve attracted an impressive list of humanitarian agencies tracking the feed who attest to its impact on the ground.

As the Haiti deployment goes forward, one of the most valuable things to do is learn from the operations and a study of the organization as it evolves in real-time. This knowledge is crucial if we want to put together successful instances in the future. Hivos is making it possible for us to hire video journalist Jon Shuler to capture the behind-the-scenes story. Jon is talking to David Kobia, Ushahidi co-founder and lead developer, Brian Herbert 4636 project lead, Katie Stanton, our contact at the U.S. Department of State, and then Patrick Meier crisis mapping lead. Finally, he is visiting the main situation room in Boston.

Ushahidi Haiti from Ushahidi on Vimeo.

[See the video and write-up here]

At Ushahidi, we subscribe to the theory that, “if it works in Africa, it will work anywhere,” and that is now proving true. The deployment in Haiti is an amazing example of how software that was created by Africans for use in Africa is being exported to the rest of the world. Documentation of this effort will help us and others replicate what Ushahidi does in the future – and it’s a rare opportunity.

Supporting African Tech Growth

Beyond Haiti, Jon will travel to Kenya in March / April to witness how real-life users in Africa are going about using the Ushahidi platform. In addition, he will cover the evolution of Ushahidi’s innovation hub – the iHub – in Nairobi, a project that will also benefit from Hivos’ support. This differs from Jon’s Haiti work, as here in East Africa his role will be to primarily interview end-users, the people on the ground using the tool, pulling out valuable information and lessons-learned that are hard to capture on our own in an objective way.

The iHub in Kenya was created to be a space for the tech community in Nairobi to use as a physical nexus space for meetings, events and work. It’s an open space for the technologists, investors, tech companies and hackers in the area. This space is a tech community facility with a focus on young entrepreneurs, web and mobile phone programmers and designers. It also serves as Ushahidi’s central hub for East African operations. Much of Ushahidi was built by volunteer developers and designers in Kenya, and it’s this community that we are giving back to, while also serving our own needs in the region.

By creating the iHub, Ushahidi will establish firm roots in Kenya, the place where Ushahidi was born and the only country in the world where we will have over a dozen installations running by the end of 2009. The iHub will service Ushahidi’s in-depth testing and deployments in Kenya, and provide a space to host Ushahidi events. With a buzzing tech scene and an emerging pool of talent in the region, Kenya offers a fertile environment to build the Ushahidi ecosystem.

The Washington post, the largest newspaper in Washington DC is currently using the Ushahidi platform to map snow clean up efforts following the massive blizzard that hit the East coast of the US.

Check it out by clicking on the graphic below.

Hat tip Dan Sinker and Andy Carvin!

UPDATE: Since this graphic was published, a few additional clarifications have come to light. Please see Robert’s comment for more details.

Shortly after we posted the original Project 4636 info graphic, a few folks involved in the project got in touch to see if we could clarify the process. There are a lot of moving parts,  many of which are constantly changing, and so the original graphic didn’t quite reflect the exact process as well as it could have. With that in mind, we worked with Josh Nesbit of Frontline SMS Medic and Nicolás di Tada of InSTEDD to make sure the graphic reflected the process as accurately as possible. The biggest update that we made is that InSTEDD’s Nuntium SMS Gateway and the Thomson Reuters Foundation Emergency Information System are now the first entities that receive and process incoming SMS’s.  Everything else is pretty much the same.

The Updated Overview

The Updated Graphic

Click the image to view the high res version.

Thanks again to Josh, Nicolás and all the Crisis Mappers for the great feedback!

Ushahidi Ghana Team

We Ushahidi Ghana are organizing a meet up this saturday to set up an instance to monitor traffic situations in Accra. This is something we planned working on during our last meet up session. Aside that, we will be hacking on the Ushahidi engine as way  to get interested developers started with the project.

If you are in Accra and want to be part of the meet up, feel free to join us at Avenida hotel on the Kojo Thompson Road, right adjacent to IPMC head office at 10am. Click here to view the location on a map

Look forward to meeting everyone.

This is a guest post by Jonathan Shuler, the multimedia journalist behind our latest video profiling some of  the people behind the haiti.ushahidi.com.

Information to save lives

It’s three o’clock on a January afternoon and I’m running around the basement of the Fletcher School at Tufts University on the outskirts of Boston.  I’m trying to keep up with Carol, a grad student in dark rimmed glasses, a down vest, and fingered gloves who is rapidly eliminating possible interview locations for me.  She, like most people I have met today, seems tired and worn out, but energized by something.  Classes haven’t started yet, but the manila walls are buzzing with activity in the handful of underground rooms that have been re-christened with names like “the Command Room” and “the Map Room.” This is the makeshift HQ for the mapping section of the Ushahidi/Haiti program.

We stride past dozens of new volunteers learning how to put information into the Haiti Ushahidi instance after confirmation.  Veterans, some with as much as a week of experience, help the greenhorns figure out how to interpret the data. There is an urgency on everyone’s face.  Despite the thousands of miles that separate Haiti from these Bostonian students, there is weighty reality: This information could save someone’s life.  Glancing at them as Carol and I roll past, I see the new volunteers with their laptops, crunching on information over the basic college wifi network and I feel like this seems so elegant and simple.  Ushahidi, now, feels obvious.

Getting the big picture

About five years ago I spent a few weeks in Sri Lanka trying to help rebuild after the Tsunami.  We got there in March so most of the real chaos had died down, but all along A2–the highway that hugs the Indian Ocean south of Columbo–the devastation still felt fresh and raw.  People where struggling to put their lives back together. Well meaning people like me where trying to come in and help, but it was clear that even the best of us were short of information; we where making mistakes.  Whole tent cities seemed to be completely unoccupied, despite the proud “Tents provided by….” sign.  Where were the people?

When you are trying to effect change during a crisis it’s hard to think about what other groups on the ground are doing or if your are working at cross purposes.  It’s even harder to know what is happening in places where you do not have people.  In Sri Lanka tent cities were built, budgets where laid out, clean water hauled in, back hows contracted, everyone trying to react to the issues as they knew them based on the information they had.  As someone who was primarily an observer, it was a well meant and noble effort, but it struggled to affect the change in aimed for.  Our forward team spent a month on the ground trying to get a fix on the situation and they did a great job, but they still got things wrong.

There was no way the team could have gotten the big picture.  The devastation was too big for a couple of guys to ingest.

It required a mob of understanding.

Ushahidi is that place were that Mob of Understanding comes to gain a voice, to be a witness and be witnessed.  Ushahidi takes conventional ideas about western intervention–“us” coming with our information to help “them”– and turns them on their head.

Haitians are delivering the information via basic communication tools like SMS. Haitian expats are translating and contextualizing the information for western volunteers in DC, Boston, the North West and Europe. These volunteers are then culling the information and mapping it. It’s not a geographically centralized beast, Ushahidi is not “based” anywhere.  There are no sophisticated contractual arraignments. It’s completely relationally driven (like the rest of the world.)

After about 30 minutes of trying different rooms and a quick rabbit trail to find Coffee, Carol and I decided to conduct the interviews in the rooms that the action is taking place, just like I had with Ushahidi’s developers in Atlanta two days ago as well as my interview at the State Department yesterday.  In two and half days I’ve seen the most functional end of Ushahidi–it’s developers feverishly repairing and improving the software as it’s running. I’ve met with Government officials who have been able to help plug the information Ushahidi provides into actionable results, and now I’m wrapping up with my last interview about 20 minutes before I grab a cab for the airport.

It’s about people and ownership

Sabina Carlson is a Fletcher School volunteer with the role of Diaspora Coordinator.  Her title as well as her insight into the people of Haiti betray her young undergrad smile as she explains that her job is to think long term for the Haiti Instance.  “Ultimately, it will be the Haitian Diaspora (in Haiti, in the States, and around the world) who own the reconstruction in the long term.” So her job is to plug them into Ushahidi so they can own the effort now.  It’s not about western intervention, it’s about people and ownership.

And that is why all of this feels so obvious to me, and so beautiful.

Jon Shuler is a Multimedia Journalist focusing on humanity during and post
crisis.  Utilizing photography and video, Jon has traveled extensively
covering such topics as Trauma Healing in the Democratic Republic of
Congo, the South Asian Tsunami, and Balkan Stability. You can see more of his work at www.jonathanshuler.com.

Hot on the heals of Brian’s excellent summary of the 4636 Project development efforts, I’d like to join in with a little info-graphic of sorts. My goal in putting this together is to present an easy-to-understand “big-picture” graphic that illustrates how a simple SMS, sent from a Haitian in need, can be transformed into a powerful resource that fuels the crisis response and recovery effort.

A Quick Recap of Project 4636

And here’s the full graphic:

Click the image to see the high-res version.

The thing that impresses me most about the whole project is how it all came together: lots of people working together across lots of different organizations. I really liked what Andrew Turner had to say about the level of collaboration that was going on all fronts, not just Project 4636:

Andrew Turner: "seeing things being created and incorporated in hours what would have taken months. human spirit and camaraderie multiplies capability"

For someone who’s recently come from the competitive creative agency world of non-disclosure and trade secret, it’s a breath of fresh air to see this level of collaboration between individuals across organizations, and to see that collaboration play a direct role in helping those in need.

Please note that this “big-picture” graphic is just that, a “big-picture”. It does not attempt to represent ALL the organizations and people involved with the 4636 Project. There have been lots of folks who have done great work here who we can’t even begin to name.