Social media can do lots of things. It can renuite old friends, spark off new ones, create romance, and solve social problems (if enough people contribute to a cause). But now, it seems that it can predict the future.
Researchers at Indiana University Bloomington's School of Informatics and Computing developed tools to analyse sentiment and mood in Twitter posts, and correlated the resulting data to the closing values on the Dow Jones Industrial Average. They found a close correlation between the sentiments expressed in the Twittersphere and the fortunes of this venerable index.
The researchers used the OpinionFinder toolkit to measure basic mood on the Internet, providing a baseline of positive vs negative attitudes. They also used Google's Profile of Mood States, another tool that provides a nuanced analysis of moods from mined micro-blogging posts. The tool could work out whether a post was calm, alert, sure, vital, kind or happy. The researchers found that the 'calmness' rating correlated very closely with the value of the Dow three or four days hence.
When the researchers tried to use the data to predict the value of the market, they got an accuracy rating of 86.7%, compared to just 73.3% when using historical stock market data alone - and this was for data samples taken during 2008, when the financial markets were in upheaval.
This isn't the only use of social media to predict the future. Jason Harper, an economics expert at ad agency Organics Inc, used the quantity and rate of social media posts about a company to determine the success of social media campaigns as they were running, rather than after. By working out how many tweets and Facebook posts were happening, and how often, Harper could calculate the velocity of posts, which could be used to determine whether a social media campaign was going to reach its overall goals (such as sign-ups to a fan page, for example).
In fact, the more we look at social media, the more predictable the world appears to become. HP Labs scientists have used Twitter posts to predict box office revenues, for example.
How predictable could things get? Hunch uses answers to thousands of questions to determine what people will like and dislike. The company hopes that this will give rise to a new kind of search technology that uses the wisdom of crowds to tailor results that appeal to you, and which are far more accurate than the existing recommendation engines developed by Web 1.0 and 2.0 companies.
Much work still remains on these predictability claims. Larger data sets spanning longer periods are needed to truly gauge their effectiveness (something that is difficult to procure, given the relative youth of many social media sites). But if more detailed analysis bears out the initial findings, then the herd could be wiser than we thought -- and group consciousness could take on a whole new meaning.
After a Masters in Computer Science, I decided that I preferred writing about IT rather than programming. My 20-year writing career has taken me to Hong Kong and London where I've edited and written for IT, business and electronics publications. In 2002 I co-founded Futurity Media with Stewart Baines where I continue to write about a range of topics such as unified communications, cloud computing and enterprise applications.