Analysing data on social media to predict trends


Analysing data on social media to predict trends

Twitter is a medium that allows us to hang out our views on just about anything, as long as we restrict ourselves to 140 characters. Millions of such messages are ‘tweeted’ every day by people from all around the world. And, just as quickly, forgotten. Much of this ‘data’ could be used for extracting information and then, knowledge. However, this is hard, because it is difficult to extract the sentiment attached to the words.

This is where a science called ‘Sentiment Analysis’ can help.

Sentiment analysis is the science of identifying sentiment associated with the opinion expressed in a text. Associating words with their meaning is called Word Sense Disambiguation (WSD).

Words are not always understood correctly. The same word can mean different things to different people, depending on the sentiment attached to it.

Take ‘unpredictable’, for instance. In separate contexts it can have different meanings, and, hence, can convey a different sentiment. It could have a positive connotation in a sentence like “The movie was so unpredictable” or a negative one in “The steering wheel of the newly launched X series SUV is unpredictable”. In the first sentence, unpredictable is a desirable trait, while in the second, it clearly highlights a flaw.

The evolution of Internet technology has enabled users to post their needs, aspirations, and emotions seamlessly onto the web. This has created a huge amount of user-generated data, in the form of words, loaded with information.

To assess the sentiment of the text, one needs to know the meaning of the text. Researchers use a technique called machine learning to ascertain the sentiment of the text. Machine learning is a process by which a machine/computer is taught how to perform a task based on thousands of sample scenarios.

Balamurali AR, a Research Scholar at the IITB-Monash Research Academy, is working on a project titled ‘Disambiguation and Multilinguality Enabled Sentiment Analysis’, under the guidance of Professor Pushpak Bhattacharya from IIT Bombay and Professor Ingrid Zukerman from Monash University.

“My work focuses on incorporating the meaning of the text while analysing for sentiment. As a researcher, I have always been fascinated with understanding what people like or dislike. I felt I could understand them better through their likes and dislikes. Who knows, I might even be able to use this knowledge to develop things which could make their life easier and more enjoyable. My research on Sentiment Analysis is a mission to explore precisely this,” says Balamurali.
The IITB-Monash Research Academy, also known as the Academy, is a graduate research school located in Mumbai, India. It opened in 2008 as a joint venture between the Indian Institute of Technology Bombay and Monash University. Students of the Academy study for a dual PhD from both institutions, spending time in both India and Australia, with supervisors from both Monash and IITB. The establishment of the Academy marks the first time that an Australian university has set up an extensive physical presence of this kind and scale in India.

Balamurali’s research also focuses on sentiment analysis in social media. “A majority of the urban population now uses this conversational media to post information and their emotions. What I am interested in is whether data from popular sites like Twitter can be used for prediction tasks based on its sentiment content. For example, can the price of a particular company’s share be predicted at a particular instance by analysing Twitter data,” he says.

Balamurali is excited about the possibilities that sentiment analysis hold in the domain of public service. “We could use the same model to predict epidemics. For example, before the onset of an epidemic, some unique characteristics manifest among the population. Tweets like ‘Running nose, I am not feeling good’ or ‘My head is hammering’ are common. If tweets like these could be analysed to find such patterns, some agencies could use Twitter to suggest home remedies.”

Sentiment analysis is capable of opening up a whole new world in how data can be mined to engender mutually beneficial engagement and interaction among people. And, it is researchers like Balamurali who are helping to make this possible.

Research scholar: Balamurali AR, IITB-Monash Research Academy

Project title: Disambiguation and Multilinguality Enabled Sentiment Analysis

Supervisors: Professor Pushpak Bhattacharyya, IIT Bombay and Professor Ingrid Zukerman, Monash University

Contact details:

The above story was written by Mr Krishna Warrier based on inputs from the research student and IITB-Monash Research Academy. Copyright IITB-Monash Research Academy.