.png)
The discovery of emerging trends has become increasingly important in recent years. Product development and innovation executives are constantly searching for ways to predict what consumers will want before their competitors. In this post, we will explore how trend discovery was done in the past, and more importantly, we will highlight cutting-edge Natural Language Processing technology that can help you identify emerging trends before they become mainstream.
The process of discovering consumer trends has undergone a massive transformation over the past 15 years, primarily due to the advent of social media and Artificial Intelligence.
In the past, companies had to rely on traditional market research methods that were time-consuming and expensive. However, with the rise of social media listening tools, it has become much easier for companies to track and analyse consumer behaviour, preferences, and opinions. In this post, we will explore the challenges faced by companies in discovering consumer trends 15 years ago, compared to the ease with which it can be done now.
To be more specific, in the past, companies relied primarily on surveys and focus groups to understand their customers. These traditional methods were often expensive, time-consuming, and had a limited sample size. Companies had to go through a rigorous process of recruiting participants, conducting the survey or focus group, analysing the data, and then interpreting the findings. This entire process could take weeks or even months to complete, making it difficult for companies to innovate and be competitive.
Moreover, surveys and focus groups were often limited to a specific geographic area or demographic, making it difficult to get a broad understanding of consumer behaviour. This lack of data often led to companies making assumptions about their customers' preferences, which could result in costly mistakes.
According to various studies, the failure rate for new products is estimated to be between 70% and 90%. In other words, most new products that are launched fail to achieve their business objectives, such as generating sufficient revenue or profitability. This underscores the importance of conducting thorough market research, testing, and analysis before launching a new product to increase the chances of success.
With the rise of social media, companies now have access to a wealth of data that can be used to uncover consumer trends. Social media platforms like Facebook, Twitter, and Instagram have billions of users, and each one of them is creating content, sharing opinions, and engaging with brands. Social media listening tools have made it easier for companies to monitor these conversations and extract meaningful insights.
Social media listening tools allow companies to track specific keywords and hashtags related to their brand or industry. These tools analyse the data and provide valuable insights, such as sentiment analysis, conversation drivers, engagement metrics and virality. This information can be used to identify emerging trends, monitor brand reputation, and engage with customers in real-time.
In addition, social media listening tools allow companies to track their competitors' activities, which can provide valuable insights into their marketing strategies and product development. By monitoring their competitors, companies can identify gaps in the market, and create products or services that meet the needs of their customers.
Furthermore, social media listening tools have made it possible for companies to connect with their customers in a more personalized way. By monitoring social media conversations, companies can identify individuals who are influential in their industry or have a large following. These individuals – the influencers - can be targeted to become brand advocates or ambassadors and to propagate offers, which can lead to increased engagement and more sales.
listening247 has developed a proprietary approach to discovering emerging trends that involves the following steps:
By using this approach, listening247 provides valuable insights into emerging consumer trends that can help companies stay ahead of the competition and better understand their customers.
The process of discovering consumer trends has evolved significantly over the past 15 years, thanks to the rise of social media listening tools such as our Social Listening and Analytics Solution. These tools have made it easier for companies to monitor and analyse consumer behaviour, preferences, and opinions. They have also provided valuable insights into competitors' activities and enabled companies to connect with their customers in a more personalized way. With the help of social media listening tools, companies can stay ahead of the competition and create products or services that meet the evolving needs of their customers.
.png){kind=tracking}
According to research by Forrester, 53% of companies worldwide have a dedicated CX department, while the remaining companies may integrate CX responsibilities into other departments, such as marketing or operations, or may not have a CX function at all. In some cases, customer care or customer service may be the only CX related function, but this setup often falls short of the lofty goals of optimizing the overall customer experience.
Customer experience (CX) and insights are both critical components of understanding and improving customer satisfaction. While they may be closely related, they are distinct disciplines that require different approaches and skill sets. Therefore, it's essential to have clarity about the ownership and responsibilities of these functions, particularly when it comes to measuring customer experience with NPS trackers and by analysing customer calls and messages.
CX is about creating and delivering an exceptional experience for the customer throughout their journey with the company. CX teams focus on understanding customer needs, pain points, and behaviours to design and optimise the customer journey. They collect and analyse data from various sources, such as surveys, customer feedback via contact centres, and predictive analytics, to identify areas of improvement and create strategies to enhance the customer experience.
CXM – a popular acronym used in this context - stands for customer experience measurement or customer experience management. When it comes to the latter there is no doubt that the CX team is responsible for it. When it comes to measuring though the Insights team is well positioned to offer support or even own it.
Customer Experience (CX) teams are primarily focused on identifying actionable insights at the individual customer level. They typically rank customer pain points based on their frequency of occurrence and then identify both proactive and reactive solutions to address them.
On the other hand, insights teams are responsible for gathering and analysing data to generate insights that can drive business decisions. Insights teams use a wide range of data sources, including customer data, market research, and internal business data to identify trends and patterns, support new product development, monitor business performance, and generally inform decision-making.
Insights teams are primarily focused on discovering strategic insights that are actionable at the total market level, rather than the individual customer level.
The process of discovering a true market insight is not straightforward. It requires multiple sources of data to be integrated, an actionable hypothesis supported by synthesised data, and a little intuition and gut feeling.
When it comes to NPS trackers, the lines between CX and insights can get blurred. NPS (Net Promoter Score) is a widely used metric for measuring customer loyalty and satisfaction. It involves asking customers how likely they are to recommend the company to others, on a scale of 0 to 10. The NPS score is calculated by subtracting the percentage of detractors (0-6) from the percentage of promoters (9-10) while ignoring the passives (7 & 8). The score provides a benchmark for how well the company is meeting customer needs and expectations.
Both CX and insights teams can benefit from NPS data. CX teams can use the score to understand how customers perceive the company and its products/services and identify areas for improvement in the customer journey. Insights teams can use the data to track overall customer satisfaction and loyalty, compare the company's performance against competitors, and identify factors that influence customer behaviour.
So, who should own the NPS tracker if CX is a separate department? The answer may vary depending on the company's size, structure, and culture. In some cases, CX and insights functions may be combined, and one team may be responsible for both functions. In other cases, the teams may be separate, and the ownership of the NPS tracker may depend on the purpose and goals of the survey or simply where the budget sits.
CX teams, if they have the skillset, could take the lead in designing and implementing NPS surveys since they are more closely related to the customer experience. Dedicated CX teams should have the expertise and experience to design surveys that capture customer feedback effectively, analyse the results, and translate them into actionable insights for the business.
However, insights teams can also play a crucial role in analysing and interpreting NPS data. Insights teams have a broader perspective on the business and can provide valuable insights into how customer satisfaction and loyalty relate to other business metrics. Insights teams can also identify trends and patterns in the data that can inform strategic decisions.
At listening247 we published a lot of articles on the importance of not just relying on a sample of customers who agree to take a survey but listening to all customer interactions using AI for natural language processing.
Ultimately, the success of a company's CX and insights functions depends on collaboration and communication between the teams. Both functions are essential for understanding and improving the customer experience, and both have a role to play in measuring customer satisfaction with NPS trackers. The roles of the two functions have to be well defined to avoid confusion and misunderstandings.
By working together, CX and insights teams can ensure that the NPS data integrated with all the other customer interactions tagged with sentiment (such as phone calls, chats, emails etc.) are used effectively to drive business decisions that benefit both the customer and the company.
.png)
AI can boost Call Centre Efficiency and Customer Satisfaction.
Call centres are a vital part of many businesses, providing customer support and assistance. In recent years, the use of artificial intelligence (AI) has become increasingly prevalent in the call centre industry, and for good reason. By leveraging AI, call centres can increase revenue and improve their overall performance.
Throughout the rest of this blog post we will use “Contact Centres” instead of “Call Centres” as it is a more appropriate description of what these organisations do. They do not just respond to calls but also to chat messages, emails and sometimes even social media posts.
Call centres can leverage AI in a variety of ways to reduce cost and even increase their revenue. By automating customer interactions, using predictive analytics, improving call routing, and analysing customer feedback, call centres can improve their efficiency, reduce costs, and provide better customer service. As AI technology continues to advance, the opportunities for call centres to use it to increase revenue will only grow.
listening247 uses proprietary machine learning models on its AI platform that are customised for each subject or product category, achieving a minimum accuracy of over 80% each time, in any language. Often, the accuracy is over 90%, depending on the amount of training data used for the custom models.
While there are ML models available for anyone to use (e.g. open source, Google, AWS and Microsoft), free or paid, the problem with those is that they are generic to a language, which means not specific to a product category. Thus, they can never reach acceptable accuracies without custom training data as top-up. Typically, their accuracies linger below 70% at best and usually around 50%-60%.
.png)
On September 7th we gave the report below to our PR agency and asked them to publish what our analysis of online posts was telling us about Liz Truss.
Sadly, the Queen passed away the next day, so the news cycle moved on from the PM’s election.
The report below was never made public but we decided to post it on our blog and in the form of a Medium article almost a month and a half later as it is an illuminating illustration of the kind of robust AI driven “social intelligence” that is now possible – you can check, at the bottom of the report, the actual posts that were shared.
-----------------------------------------------------------------
This is the 4th report produced by listening247 with data gathered from social media and other public online sources between July 11 and September 6 2022 on Politicians. The 1st report was based on data collected between Sept. 15 and Dec. 14, 2021. The 2nd Dec. 15, 2021 to March 15, 2022 and the 3rd March 16 to July 10, 2022.
The sources of posts included in this report are Twitter, news, blogs, forums, reviews and video.
It is important to make a strong distinction between polls/surveys which are based on a sample of respondents – plenty of those exist and they are not necessarily accurate - and social intelligence which is not based on a sample - this is a unique report and the first of its kind. The data collected by listening247's CXM as well as Social Intelligence and Text Analytics platform are based on the universe of all the posts about the names included in the research (not a sample of posts) and it is the unsolicited opinion of the people who posted these posts – in contrast to polls they were not paid to answer questions, so they had no incentive to cheat or write an opinion that was not theirs.
The main KPIs used to rank the subjects of this research were:
In the table below the 3 politicians are ranked based on total number of posts from all sources.
It is now the 3rd time that the total number of posts or share of voice is predictive of who will win an election.
In the previous report we published Rishi was leading in this metric and he was the one elected with the highest number of MPs. Now from July 11 to September 5th the tables turned. Liz has more than double as many posts as Rishi and won the vote of the 170,000 conservative members.
In terms of net sentiment score during the 24 hours post Liz’s election the ranking is almost turned on its head.
Keir has positive 3% whilst Rishi Sunak is closer to him with negative 3% and Liz has negative 8% a whole 11% worse score in public sentiment than the leader of the Labour Party.
The examples of posts below are a representative sample of what most people post about the next national election in January 2025:
It is quite clear that they all think Liz was the worse PM to go against Keir.
Three numbers are very important to keep in mind and understand what they represent:
The two final candidates (Liz+Rishi) of the conservative PM race were selected in multiple voting iterations by ~350 conservative MPs.
On September 5th ~170,000 conservative members voted and elected Liz Truss as the Prime Minister.
The total electorate for parliamentary votes in the UK has over 46 million voters.
Our report reflects the opinions of the 46 million voters; thus, the ranking may be able to predict what would happen if the vote for the election of a conservative PM was put to a national vote yesterday. To better interpret our rankings above we should keep in mind that in 2022 around 85% of the UK population are social media users.
From that we can infer that the online posts gathered from various sources between March and September this year may impact up to 85% of the voters; it could be a bit less because the older population who do not have access to social media are all voters whereas the 85% (people with access to social media) includes children below 18 who are not voters yet. Having said that online News is one of the sources (the media) which is editorial and impacts the opinions of everyone who are exposed to the media.
The discussion in our previous report about possible scenarios was inferring that the conservative members should pay attention who was the most likely candidate to win the national election in January 2025 and let that inform their decision.
Unfortunately, they did not do that.
The social media data indicates that Rishi would have a better chance to beat Keir.
The question now is if this data was predictive for the last two finalists and for the September 5th vote - which it was - will it also be predictive for the national election results in 2025? It looks like it is when 2 out of 2 times the share of voice predicted the outcome!
Stay tuned for more data from listening247 on the subject.
----------------------------------------------------------------
This is what we wrote on September 7th, 2022. I think for the 4th time unsolicited citizen opinion from social media and other online sources proves to be much more predictive than polls which are based on samples of respondents (who sometimes lie and sometimes forget what they said or did a week ago) that may not be representative.
We think that it is about time for social media listening and analytics to take its rightful place in the political forecasting business.
.png)
There is a relatively simple formula which describes “weak” or “narrow” artificial intelligence: AI = ML+TD+HITL. To be more specific, this is the definition of supervised machine learning, which is the most common method to produce artificial intelligence. The acronyms in the formula stand for:
Strong artificial intelligence - as defined by the Turing test - is when a human has a conversation with a machine and cannot tell it was not a human, based on the way it responds to questions. The optimists believe that strong AI is 10-15 years away whilst the realists/pessimists say not before the end of this century.
Over 90% of all human knowledge accumulated since the beginning of time, is unstructured data. That is text, images, audio, or video. The other 10% are numbers in tables which is what quantitative market researchers usually use. The qualies, they are the ones using unstructured data, but the volume is limited to a few pages or a few video clips that a person can read/watch in a couple of days.
Other than reading, listening to, or viewing unstructured data, 15 years ago there was no other way to discover their content and understand their meaning. Thankfully (especially if we are dealing with big data) today there is a way to discover and understand the information hidden in mega-, giga-, tera- or n-ta-bytes of data; you guessed it, it is AI. Machine learning allows us to create models that can process large files of text or images in seconds, and annotate sentences, paragraphs, sections, objects, or even whole documents with topics, sentiment and specific emotions. Sentiment and semantic analysis are the two most popular ways to analyse and understand unstructured data with the use of machine learning or a rules based approach. When the unstructured data to be analysed is in text format, the discipline falls under Computer Science (not linguistics funnily enough) and is called Natural Language Processing (NLP) or Text Analytics.
Semi-supervised-, unsupervised- and deep-learning are other forms of machine learning, used to a smaller extent in a market research context, even though deep learning implementation is picking up speed - especially for image analytics.
There is a multitude of users, data sources and use cases within an organisation. Let’s take a look at relevant data sources first:
ESOMAR mainly caters to the market researchers in organisations globally, but there are many more users of text and image analytics solutions sitting in different departments, that can benefit from using AI to understand unstructured data. Here is a combined list of users and use case examples for each one, which is not exhaustive by any means:
If we agree that social intelligence is currently the most popular application of AI in research and insights then it does make sense to review possible questions that can be answered using it.
If you are amenable to a bold statement such as “social intelligence may replace some traditional market research methods used to solicit consumer opinions” then here is a list to consider:
Of course whether social intelligence can replace them altogether or enhance them depends on the country, language and product category. If you have not embraced the use of AI yet, to tap into the wealth of unstructured data available to us everywhere, then at least keep an open mind and keep asking questions that will help you make an informed decision when the right time comes.
.png)
listening247 answer:
listening247 is a technology company in the market research sector offering platform access as well as end-to-end market research services to Agencies, FMCG, Retail, Financial Services, Telecoms, Tourism & Hospitality, Healthcare, Automotive, Government & NGOs.
listening247 answer:
listening247 answer:
The pricing for social intelligence is based on product category, language (not country) and period covered. A rule of thumb is that an average product category is defined by up to 12 competitive brands. These 12 brands are used as keywords for harvesting from the web. The frequency of reporting and the delivery mechanism also have an impact on cost.
The pricing for any text or image analytics processing and annotation through an API, regardless of data source, is charged per annotated post or image.
listening247 answer:
Yes, for many different product categories and languages and in different formats e.g. PDF decks, infographics, one pagers and demo dashboards.
listening247 answer:
For Social Intelligence listening247 harvests data from social media and any public website such as Twitter, Blogs, Forums, Reviews, Videos, News and also Facebook and Instagram with some limitations that apply to all data providers.
The listening247 text and image analytics technology is source agnostic and can therefore ingest client data from open ended questions in surveys, transcripts of qualitative research, call centre conversations or any other source of unstructured data.
listening247 answer:
For social intelligence listening247 uses all the available methods to harvest data from public sources i.e. direct APIs, Aggregator APIs, Custom crawlers and scrapers, RSS feeds etc. When doing so listening247 abides by the ESOMAR code of conduct, the law and the Terms & Conditions of the sources.
For client data - see answer to Q6 - the client can share its own data by email, on FTP, on cloud drives or through APIs.
listening247 answer:
For social intelligence yes - as long as the posts still exist online at the time of harvest.
listening247 answer:
Text, images, audio and video can be harvested from the web or taken from other sources (see answer to Q5). listening247 - the listening247 software - does offer the capability of data harvesting from online sources. It provides buzz (word counts), sentiment, 7 pairs of opposite emotions such as ‘Love Vs Hate’, and semantic (topic) analysis. The topic analysis provided is inductive (bottom-up) and top down. Topics can be broken down in sub-topics and sub-topics in attributes and so on. listening247 can also analyse images for objects, brand logos, text (extraction) and image theme (aption). It uses 3rd party technology to turn audio to text, followed by its own text analytics capability to analyse for sentiment, emotions and topics.
listening247 answer:
The listening247 software represents the implementation of years of R&D funded by the UK government and the EU. It includes supervised, semi-supervised and unsupervised machine learning as well as deep learning for data “cleaning”, sentiment, emotions, topics and image annotations. For data “cleaning” and topic annotations listening247 uses a combination of engineered approaches and machine learning. All listening247 custom models and set-ups continuously improve their accuracy. The user can also provide improvements to the supervised machine learning models by adding training data any time.
listening247 answer:
The text analysis is done at document, paragraph, sentence, phrase, or keyword mention
level. This is the choice of the client. The analysis extracts named entities, pattern-defined expressions, topics and themes, aspects (of an entity or topic), or relationships and attributes – and it offers feature resolution, that is, identifying multiple features that are essentially the same thing as the example in the guidance (Winston Churchill, Mr. Churchill, the Prime Minister are a single individual.)
The sentiment or emotions analysis is ascribed to each of the resolved features or at some other level; the user may choose the resolution of e.g. sentiment/emotion and semantic annotation.
listening247 answer:
listening247 provides document level data with the capability to drill through to the posts/verbatims, making it possible for users to verify the accuracy of all the annotations made by the models.
listening247 answer:
In literally all languages, including the likes of Arabish (Arabic expressed in Latin characters) and Greeglish (Greek expressed in Latin characters), since the automated analyses are done using custom models specifically created for the particular product category and language. The only trade-off is that it takes 1-3 weeks to create the set-up that guarantees the accuracy as advertised.
listening247 answer:
listening247 uses its own proprietary software and models to produce all the analyses. It provides fully configured customised models; the end user is not responsible for that training but has the option to participate or improve if they wish to do so.
listening247 answer:
When it comes to social intelligence, limited demographics are available in the meta-data of normally harvested posts - see Q6. Any and all demographics can be inferred/predicted using a custom machine learning model which is trained to classify authors based on the way they write. The accuracy of prediction can be validated by testing it on new annotated data that was not used to train the model.
listening247 answer:
For social intelligence listening247 typically harvests and reports all the posts from all the keywords and sources included. This is called census data as opposed to sample data. Data sampling is only done at the training data generation part of the process when the approach used is supervised machine learning. A random sample of 10% or up to 20,000 posts whichever is smaller is used as training data annotated by humans.
When it comes to sources other than the web, lower samples are needed to train the machine learning algorithms in order to reach the minimum accuracy.
listening247 answer:
listening247 was originally designed for market research purposes (in any language) thus the focus is on data accuracy and data integration with other sources such as surveys and transactional/behavioral data for insights. A few years down the line, it is now also being used for sales lead generation and identification of micro/nano influencers.
listening247 answer:
For social intelligence, listening247 uses a combination of boolean logic and machine learning models to eliminate irrelevant posts due to homonyms. The priority and focus during the set-up period of a social listening tracker is to include all the synonyms (also misspellings, plurals etc) and exclude all the homonyms. Typically the data processed is over 90% relevant i.e. only a maximum of 10% is noise.
listening247 answer:
listening247 offers a money back guarantee for the following precisions in any language:
Recall is usually at similar levels but it is not deemed as important as precision for market research purposes because if we end up with say 50% of all the data (50% recall) the sample is still hundreds if not thousands of times higher than the samples we use to represent populations in surveys.
For image captioning the committed Bleu-1 score is >75%
listening247 answer:
Yes
listening247 answer:
Different users have different definitions of spam. These are identified at the beginning of the project and eliminated during the set-up process described under Q17 by using a combination of boolean logic queries and custom machine learning models. Clients are also enabled to flag and remove spam themselves should they find any.
listening247 answer:
Yes absolutely. Even more than that since listening247 complies with the ESOMAR code of conduct which is stricter than the local laws.
listening247 answer:
listening247 abides by the ESOMAR code of conduct and not only stays informed about changes with the laws and terms & conditions of specific sources it actually gets actively involved in making sure the clients/users of these services stay well informed (e.g. the initiative to create this document under the auspices of ESOMAR). listening247 uses the highest standards of security in storing and transmitting data.
listening247 answer:
The codes of conduct and industry standards including the ICC/ESOMAR International Code on Market, Opinion and Social Research and Data Analytics; the Market Research Society in the UK (MRS).
listening247 answer:
By abiding to the codes of conduct mentioned in Q23. In the occasions when an author of a post is contacted by listening247 the etiquette of the medium where the post was found is strictly followed and the medium/platform allows such contact and is usually expected by the authors of such posts. No offers are made unless the author indicates acceptance in the process of following the contact etiquette.
listening247 answer:
Only data from public sources are shared with users without masking. If the data is not from a public source then it is only offered in aggregated form or masked.
listening247 answer:
Most of the data in social intelligence is public but in the occasions when the data is owned by the client or is sourced from a non-public source cutting edge security measures are used. listening247 uses secure sites and encrypted transmissions to protect the data in its custody.
All the communication from and to listening247 happens through a Secure Sockets Layer (SSL) to ensure the encryption of communication client-server. In addition our hosting partner has successfully completed multiple SAS70 Type II audits, and now publishes a Service Organization Controls 1 (SOC 1), Type 2 report, published under both the SSAE 16 and the ISAE 3402 professional standards as well as a Service Organization Controls 2 (SOC 2) report. In addition a PCI (Payment Card Industry) DSS (Data Security Standard) Level 1 certificate has also been received. The users are welcome to carry out their own audits.
.png)
Humans are inclined to think linearly. We overestimate the short-term and underestimate the long-term when we forecast. Quite often, linear trends are interrupted by “hockey sticks” that no-one could foresee. For example, no one predicted the exponential growth of computers and smartphones but we have been forecasting flying cars for over 30 years.
Credible forecasters in their quest to become better, look at their past predictions and measure their “batting rate”. This is what Greenbook asked me to do with my 2015 predictions about market research.
There is one dynamic about forecasting that needs to be explained. It may sound like an excuse for getting a prediction wrong but really it’s a compliment to the forecaster. When a prediction puts governments, companies or people in a bad place in the future, then the affected parties do their utmost to avoid that future. Case in point: blockchains were predicted to be the end of traditional banks as we know them due to their power of disintermediation; according to a FinTech rep at a conference in London, Bank of America owns 82% of all blockchain related patents in the US. This article does not mention the percentage but it quotes BofA as the leading company with the most patents in the field!!!
Happening: 4
Not Happening (yet): 4
Possibly happening: 2
Two years later - in 2017 - I published a new list of predictions. As you will see below the quality of my forecasting improved a lot.
Getting the hang of it!
Happening: 7
Not Happening (yet): 1
Possibly happening: 2
From 40% to 70% in two years…. not bad if I may say so myself :)!
One last prediction (from Reuters this time) which we endorse: The social analytics market will be US$16 billion by 2023.
Unlike one of my recent blog posts titled Social Listening - Social Analytics - Social Intelligence, the 3 bigrams in the sub-heading are not part of a continuum, they are synonyms.
Synonyms are words that do not necessarily look or sound alike, but they have more or less the same meaning, while homonyms are words which are spelled the same, although they mean different things.
For the social intelligence discipline, synonyms and homonyms are treated in a diametrically opposite manner: the former are included when gathering online posts whilst the latter must be excluded; failure to do so results in another bigram we so often use in the data analytics business: “Garbage-in…”!
Sometimes, depending on the popularity of the homonyms, more than 80% of the posts gathered - using a social media monitoring tool are irrelevant - referred to as “noise” (as opposed to signal). Only if we have a way to remove the noise can we avoid completing the popular saying mentioned in the previous paragraph with: “…Garbage-out”.
But I digress…
This post is about efficient and meaningful ways to integrate unstructured and eventually structured data sources as part of an organisation's customer experience (CX) measurement or customer management (CM) process - a relatively new more encompassing term gaining ground on CX - in order to discover actionable insights.
This new process of beneficial unstructured data fusion from multiple source types can be described in the following 8 steps:
First a quick reminder as to what constitutes unstructured data:
Text analytics is the easiest to perform (as opposed to audio analytics for example) hence the idea to transform all forms of unstructured data to text for easier manipulation.
One of the most useful sources of unstructured data for businesses is their call center audio recordings, with conversations between customers and customer care employees. These audio files can easily be transformed to text (voice-to-text) using specialised language specific machine learning models. An accuracy metric used for the transcripts produced is WER=Word Error Rate which should be lower than 10%.
Another popular source of insights are images e.g. posted on social media or shared on a business client community. A deep learning model adequately customised can produce a caption describing in text what is illustrated in each image (image-to-text).
When it comes to video, a combination of voice-to-text and image-to-text tech can be used.
When all sources of unstructured data are turned into text, they then need to be uploaded onto a text analytics platform, usually in the form of a JSON or CSV file.
If the same platform has the capability to provide data from additional sources, such as online posts (text and images) from Twitter, Facebook, Instagram, YouTube, reviews, forums, blogs, news etc. so much the better. It can serve as both a social intelligence and text analytics platform.
If needed, text from each source type can be uploaded or gathered and saved separately and merged at a later stage, so as to take a bespoke approach to cleaning and subsequently annotating the text using custom machine learning models for each source.
When it comes to client/user owned data they are all intrinsically clean (read relevant) since the source types are:
As for the data gathered from online sources – what is commonly known as social listening or social media monitoring – that is where a thorough data cleaning process is required. The problem as already indicated above is the homonyms. When a Boolean logic query is created to gather posts from social media and other public online sources, using a brand name like Apple or Coke or Orange as a keyword invites a lot of “noise” as you can imagine. The platform is required to offer easy ways to eliminate posts about apple the fruit, cocaine and orange the colour or fruit..
There are two ways to get rid of the irrelevant posts which sometimes make up more than 80% of all posts gathered.
If data cleaning is done properly, we can expect brand/keyword relevance over 90%.
Natural language processing is the umbrella discipline that takes care of this step in the process. Ideally the use of machine learning models to annotate text in any language works best, but sometimes a rules-based approach may be a shortcut to enhancing the annotation accuracy.
A good text analytics tool offers multiple options i.e. the ability to train generic & custom unsupervised machine learning models or using native language speakers as well as a taxonomy creation feature using a rules based approach.
Text can be annotated for sentiment, topics, relevance, age or other demographics of the author (if not otherwise obtainable), customer journey etc. A minimum accuracy of annotation should be declared and aimed for, and the users need to be able to easily verify the annotation accuracy themselves.
This step can happen before or after the merging of the various data sources, depending on their homogeneity.
For the longest time data fusion or integration or merging from different sources meant weeks or months of data harmonisation, so that the different sources could fit together and make sense. Merging 5-10 source types of unstructured data after steps 1-3 above only takes a few minutes, not months. It would take a few hours from start to fusion.
A powerful filtering tool is required for the user (data analyst) to be able to drill down into the data and discover interesting customer stories which might lead to actionable customer insights. For example, the user could first filter for negative sentiment, then for a specific brand, after that a topic and finally a source type before they start reading individual interactions to get an in-depth understanding of the WHY and the SO WHAT.
Once the data is cleaned, merged, annotated, and explored, it can be delivered in multiple ways such as:
Data visualisation via PowerPoint slides, drill down or query dashboards and alerts work best. Ideally the data formats should be flexible so that they can work with multiple data visualisation tools.
For now, data fusion included in a CX/CM program is a better fit for larger corporations, for two reasons:
Hopefully soon there will be versions of SaaS products that will make this process efficient and inexpensive enough for SMEs (SMBs) to be able to afford it.
That is what we call the democratisation of data analytics and market research.
The more data sources we integrate the more likely it is for a data analyst, the user of a tool such as listening247, to be able to synthesize actionable insights in their true meaning.
It seems that the biggest gain from this newly found ability to accurately annotate text in any language and fuse/integrate/merge from any source type in a matter of hours is in the discipline of customer experience (CX) measurement and management (CM).
CX and CM are increasingly seeking to encapsulate market research, business intelligence, customer care and other business disciplines and are meant to perfect the customer path to purchase, minimise brand defectors and maximise the number of advocates.
It was a lightbulb moment during lockdown; a few days after the team completed a piece of work, it suddenly hit me. Online posts from Twitter, Facebook, Instagram, blogs, forums, news, reviews and videos were fused with call centre audio files and survey verbatims in just 3 days, done for the first time and done right. Albeit from very different sources, involving both solicited and unsolicited opinion, this data had something in common - it was all unstructured data.
When I compare the amount of effort that is required to integrate structured data, with what we experienced integrating text and audio (unstructured data) during the “light bulb event” the contrast could not be more surprising!
If you are dealing with numbers in tables, you’re looking at column headings, product names, units and rigid time periods, so integrating various sources means that everything should be harmonised, for example:
Harmonising structured data to import it into one platform and then further manipulate it to integrate the various sources in order for meaningful analytics to be possible takes weeks, sometimes even months, compared to the 3 days to import, integrate, annotate, and explore unstructured data from various sources.
With unstructured data the integration process is simple; all data in text format can be annotated for relevance, brand, sentiment and topics in an automated way using machine learning models or taxonomies. Data in other formats (such as image or audio) can be converted into text in order for the same process to follow. This makes it possible to annotate call centre conversations or images from social media, just as easily as text in online posts and responses to open ended questions from surveys.
Fig. 1 Ingesting survey verbatims on listening247
The difference that makes all the difference (pun intended) when it comes to integrating structured vs unstructured data is that with the former the intelligence is already an added layer before the data fusion takes place, whilst with the latter the text is ingested and integrated before consistent intelligence is added to the dataset as a whole e.g. brands, sentiment/emotions and topics. Once the data is integrated it is already homogeneous (since it is all text) so it is straightforward to annotate it using custom or generic machine learning models and taxonomies - without having to worry about harmonisation.
Fig 2. Annotated online posts with brand topics and sentiment on listening247 Data Explorer
There are some obstacles to integrating and annotating unstructured data other than text such as audio that needs to be transcribed and images that need to be captioned with text; only when that happens can the accurate annotation of all the integrated data sources take place. There are even more obstacles if the data to be fused involves multiple languages.
Fig. 3. Image caption example, image-to-text
Thankfully, technology is available to enable voice-to-text and image-to-text transformation, as well as accurate annotations. Without accurately adding layers of intelligence, big data and especially text is not only useless, but with the wrong labels also harmful.
A data analyst cannot be expected to read millions of online posts, but what they can do is use a smart filtering tool to drill down and explore the annotated documents (e.g. social media posts or call center threads) and discover the “gold nuggets”, the elusive actionable insights.
The future of unique and actionable insights lies in data fusion of unstructured + structured data. Some of this data will belong to the companies e.g. sales data, and some they will need to procure e.g. 3rd party online posts or survey results.
Integrating unstructured data is more effortless and straightforward than you might think. You only need a good unstructured data analytics tool.
S-L-A-I
Social Listening, Social Analytics, and Social Intelligence - are they the same or are they integral parts of a sequential process, a so-called continuum?
Quite a few pundits have discussed this question in their articles, blogs, and essays. The most controversial of the three is social intelligence; if you Google it you will find its Wikipedia definition on first position explaining that it is “the capacity to know oneself and to know others”.
Of course, the alternative definition, the one that was coined after social media monitoring and analytics became popular only appears on page four of Google search - which you would only know if you are me and you search for social intelligence. In this secondary context, this bigram* means: the knowledge and insights that organisations can extract from online posts (mainly on social media platforms) published by their customers and other stakeholders.
The precondition to get to actionable insights is to avoid “Garbage-in” during the so-called data harvesting process, and to use appropriate machine learning models to maximise the accuracy of brand, topic, and sentiment annotation.
But let us look at the three bigrams one at a time,*combination of two words.
Social listening is short for social media listening, and to some a synonym of social media monitoring. Most people use social listening as an all-inclusive term for all online sources from which online posts can be gathered or harvested. However, in addition to popular social media platforms such as Twitter, Facebook, Instagram, YouTube, we also can harvest data from blogs, forums, news, and reviews. There are other sources that are country specific, such as Weibo for China and VK for Russia.
It is of paramount importance to harvest for all synonyms and avoid all homonyms (garbage-in) which can be over 80% of all harvested posts. A homonym is a word that is spelled the same way as the keyword for harvesting but means something entirely different, which makes it irrelevant for the project at hand. The classic example used to explain this problem is: wanting to harvest posts about Apple the company but ending up with lots of posts about the fruit or juice; let alone Apple Martin who is Gwyneth Paltrow’s daughter and if not excluded will invite a lot of noise in the dataset rendering it not only useless but dangerous for the user!
Social analytics is what happens after the posts are harvested from the various sources and irrelevant posts are removed as “noise”. For data analysis to take place, accurate intelligence needs to be added to the dataset using machine learning and/or rules-based NLP methods. The most common annotations added to each post or relevant snippet within the post (this is a longer story that requires its own blog post) are: brand, sentiment, emotions, and topics/subtopics/attributes.
These annotations can be added for text in any language, and the accuracy sought after – measured in precision, recall and F-score – should be over 75% in all cases. It is even possible to reach F-scores that are over 95% with focussed and context related training of suitable machine learning algorithms.
Social intelligence is the wisdom discovered by exploring the intelligent dataset. You see, adding brand, sentiment, emotion, and topic annotations to a dataset makes it “intelligent” but in order to find wisdom or “actionable insights” a lot more than just accurate annotations is required.
For the time being, an intelligent data cruncher and a powerful filtering or drill down tool is still needed to explore a dataset and find the gold nuggets.
We like to think of the listening247 unstructured data analytics platform also used for social listening, as the Google Maps of big data. It enables a user to navigate in a maze of millions of online posts - or other documents for that matter - safely and accurately from A to B; B being the destination or in our case the actionable insight. The path to finding the actionable insight is oftentimes a data story worth telling.
.png)
Most people prefer order to mess, hardly a surprising conclusion.
A score that enables ranking in multi-player environments provides order and the ultimate gamification. Ideally it should be a composite score. This is a score that combines multiple metrics in one all-encompassing index.
Gamification does not mean turning a serious activity into a game; it is using gaming techniques to provide participant motivation and make the activity more fun overall.
I do like to think of business as a game. Some people take it too seriously, to the extent that it has a pathological effect on them – it affects their health in a negative way. Business is not a life-or-death endeavour. Winning is fun, losing is dreadful (worse for bad losers like myself) but it is not the end of the world.
So, let us break down how the SPS is calculated and its benefits for brands and for individuals; starting with the business perspective.
Brands and their parent organisations are always looking for the ideal KPIs that will drive their performance. NPS was once hailed as the single metric a company needed to measure and predict its future performance. NPS stands for Net Promoter Score and it is produced via a single question to customers, usually delivered via a survey: On a scale from 0-10 how likely is it that you would recommend Brand X to your friends and colleagues?
All those who provide a score between 0-6 are considered detractors, whereas the 9s and 10s are considered promoters; the 7s and 8s are polite negatives or passives at best.
The calculation of this composite score is as follows NPS=((promoters-detractors)/all respondents) X 100, with scores ranging from -100 to +100.
A similar idea applies to the Social Presence Score (SPS) for brands; however, its calculation is not as straightforward. Several social intelligence metrics have been considered by our data scientists and our conclusion was to use the following:
The SPS can have a value between 0 and 1 (see Fig.1 above); it is calculated, for a certain period of time, as one aggregated metric for multiple brand posts; it can be offered as a single score from all source types, or for individual ones such as Twitter, Facebook, Instagram, YouTube or Tik Tok (see Fig.2 below).
There is a secret sauce that even if I wanted, I would not be able to adequately describe in a mainstream article, and that is the weighting of the above-mentioned metrics in the SPS. Not only that but also the entire process, starting before the online posts are annotated, to eliminate irrelevant posts due to homonyms i.e.clean the data and remove the noise.
The benefit of having a score like this as a brand is, not surprisingly, the ability to:
The motivation to track a social presence score is not that dissimilar for individuals. Influencers and other high profile individuals want to know how their personal brand is doing compared to others and where they rank.
The metrics we use to create the composite Social Presence Score for individuals are similar but not all the same as for brands (see Fig.3 below):
The benefits for a person to know their SPS are:
It goes without saying that even if the SPS is perfectly synthesised with its composite metrics, the accuracy of the individual metrics included must be measurable and acceptable. It is always possible to reach over 80% accuracy for sentiment, topic and brand relevance annotations.
So, what do you think?
Would you like to know your SPS in relation to others?
I know I would; I miss having one since klout score ceased to exist.
Social presence score (SPS) comes to the rescue; it is the new single KPI for brands and individuals based on all online mentions - not just a sample – that can be used to measure the overall success of their marketing efforts.
As businesses seek to understand their standing in the digital conversation, listening247's Net Sentiment Score (NSS™) emerges as a pivotal metric. This proprietary formula quantifies online sentiment towards brands, transforming raw social media data and unsolicited customer opinion into actionable insights.
Let’s deep dive into the NSS™ Score and explore its origins, how it operates, and its significance in today's digital-first world.
The Net Sentiment Score™ was developed by listening247 to fill a crucial gap in social media analytics. In a landscape saturated with diverse opinions, it provides a standardised way to assess and compare brand sentiment. This metric is the result of advanced machine learning models that meticulously annotate sentiments as positive, negative, or neutral, ensuring a nuanced understanding of the digital conversation landscape.
Simplicity lies at the heart of the NSS™'s effectiveness. By focusing on the balance between positive and negative mentions and considering the volume of discussions, the NSS™ offers a comprehensive snapshot of a brand's online health. This approach allows for the aggregation of vast amounts of data into a digestible, numerical format, empowering businesses with the clarity needed to navigate the complexities of online reputation management.
This seemingly simple calculation belies the complexity and sophistication of the technology behind it. listening247's algorithms analyse vast amounts of online content, from tweets and blog posts to forum discussions and reviews, employing natural language processing (NLP) and machine learning to accurately capture and categorise sentiments.
The process involves more than just keyword recognition; it delves into the nuances of language, picking up on context, irony, and even regional dialects to ensure the sentiments are accurately interpreted. The result is a score that ranges from -100 (entirely negative) to +100 (entirely positive), offering a clear, quantifiable measure of online sentiment.
The implications of the Net Sentiment Score™ for businesses are profound. It serves as a vital indicator for assessing the impact of social media conversations on brand perception. Here's why NSS™ is indispensable:
The Net Sentiment Score™ by listening247 represents a significant advancement in the analysis of social media sentiment. Its creation marks a strategic move towards a more informed, data-driven approach to understanding digital conversations. As businesses continue to operate in increasingly digital environments, the NSS™ offers a vital metrics for navigating the complex dynamics of online brand perception. Through its precise, insightful analysis, businesses are better equipped to foster positive engagements, adapt to consumer needs, and ultimately, drive success in the digital age.
To explore the depth of insights that the Net Sentiment Score™ can bring to your brand, we invite you to reach out and request access to your brand health dashboard. The brand health tracking dashboard offers a comprehensive view of your brand's digital presence and sentiment, enabling you to make informed decisions that drive your brand forward.
Whether you're looking to enhance your social media strategy, improve customer engagement, or simply gain a better understanding of your brand's position in the digital landscape, our team is here to guide you through the insights our dashboard can provide. Contact us today to unlock the full potential of your brand's digital narrative.
Michael Michalis, CEO of listening247, delves into how AI-powered unstructured data analysis can transform marketing strategies with greater nuance. In today’s data-driven landscape, the ability to gather, interpret, and act on data is crucial for business success. However, the sheer volume and variety of data—from meticulously organised databases to spontaneous social media posts—can be overwhelming. This data can be broadly categorized into structured and unstructured data, each with its own implications for decision-making and strategy.
The term structured data refers to data that resides in a fixed field within a file or record. Structured data is typically stored in a relational database (RDBMS). It can consist of numbers and text, and sourcing can happen automatically or manually, as long as it's within an RDBMS structure. It depends on the creation of a data model, defining what types of data to include and how to store and process it.
Unstructured data (UD) encompasses all data that lacks a predefined format or data model, making it distinct from structured data. Unlike structured data, which is neatly organized in tables and fields, UD includes diverse formats such as text, rich media, social media activity, and surveillance imagery. Its volume significantly surpasses that of structured data, making it a vast, often untapped resource for insights.
While structured data offers valuable numerical insights, it falls short in capturing the depth and nuance found in UD. This type of data, growing at an exponential rate, includes everything from social media posts to multimedia content. As IDG predicts that 93% of digital data will be unstructured by 2022, businesses that learn to harness this data will gain a competitive edge. Despite its potential, UD remains underutilized, often referred to as "dark" information due to its raw and unprocessed nature.
Recent advancements in AI and machine learning have revolutionized how unstructured data can be leveraged for marketing. By applying these technologies, companies can transform vast amounts of UD into actionable insights, such as sentiment analysis from customer reviews and feedback. This shift allows marketers to move beyond traditional demographic segmentation and gain a more sophisticated understanding of their market, revealing hidden opportunities and enhancing strategy through sophisticated "dark analytics."
In conclusion, as Michael Michalis highlights, the true value of data lies not just in its collection but in its interpretation and application. While structured data offers clarity, unstructured data holds the depth and nuance necessary for comprehensive marketing insights. Advances in AI and machine learning are making it possible to unlock this previously elusive "dark" data, providing businesses with richer, more actionable insights. Embracing these technologies can transform how companies understand and engage with their customers, revealing valuable opportunities hidden within the vast expanse of unstructured data.
Original Source: Analytic Insight
London, County, United Kingdom, October 12, 2021: A leading tech firm that specialises in CX management, social intelligence and text analytics celebrates success in winning long-term contracts with top tier organisations in Eastern Europe, the Middle East, Southeast Asia and LATAM, having recently rebranded to ‘listening247’.
Headquartered in London, listening247 was established in 2010 to help brands carry out digital market research for insights into consumer sentiment through social intelligence and online communities. In 2012 it pivoted to develop its proprietary technology in the same space and focussed for the following seven years on purposeful R&D to solve some of the biggest problems in leveraging unstructured data: the annotation in multiple languages.
Having expanded and invested in its AI capabilities to include CX measurement and management, customer journey optimisation and alternative data available on Bloomberg terminal, the firm now delivers ‘AI driven insights’ to multinationals, agencies and corporate organisations across various industry sectors ranging from FMCG and retail through to finance and telecoms. listening247’s clients are primarily data driven organisations that want to leverage customer interactions they possess from calls, private chat messages, emails surveys and social media.
Its recent success in winning multi-year contracts comes as more organisations focus on enhancing the online customer experience in the wake of the COVID-19 pandemic, as noted by listening247 founder, and CEO, Michalis Michael:
“Over the last 18 months or so, businesses across the globe have had to adjust how they operate and interact with their customers as a result of the COVID-19 pandemic. Although many countries are now pushing towards normality, times have changed and so have consumer expectations. More organisations have realised this and are utilising CX measurement and management to understand the clear gap between what their customers now want and expect Vs the product quality or level of service they are receiving. Advances in artificial intelligence enable us to ‘plug this gap’ by delivering and analysing personalised consumer insights on every single interaction to then optimise the customer journey and increase both brand loyalty resulting to business growth. With technology continuing to advance at an exponential rate, the CX management/measurement and alternative data space is proving to be incredibly interesting, and both the team and I are excited to see what the next six to twelve months will bring.”
Supported by a high calibre advisory board, including Peter Nathanial, ex-Group Chief Risk Officer for the Royal Bank of Scotland, listening247 is forecast to continue its accelerated growth trajectory over the next few years. To maintain competitive advantage, the leading AI driven insights firm will be launching its pre-series A funding round in early 2022.
Original Source: NEWS WIRES
In a data-driven world, the 'DIKW pyramid'—with its levels of data, information, knowledge, and wisdom—has long been a cornerstone of business intelligence. However, in the realm of social intelligence, this model is evolving, replacing 'wisdom' with 'insight,' a term often misused and misunderstood in the industry.
In a data-saturated world, the ‘DIKW pyramid’—which ranks data, information, knowledge, and wisdom—has become a crucial framework for business leaders seeking to add value at each level. However, in the realm of social intelligence, this model is evolving to place 'insight' at the pinnacle instead of wisdom. The term 'insight' is often misused, sometimes being equated with mere information or knowledge, which can dilute its true meaning and significance.
Fig 1. DIKW pyramid
To clarify, insight should be viewed as the apex of our revised pyramid, transcending both intelligent data and actionable information. While intelligent data and actionable information are essential, insight represents a deeper understanding that combines various data sources with intuition to deliver strategic value. Recognizing this distinction helps businesses leverage their data more effectively, ensuring that they are not just reacting to information but proactively using insights to drive long-term success.
Insight is often derived from data, but it is crucial to differentiate between raw data, intelligent data, and true insight. While adding intelligence to raw data—through annotation and quantification—enhances its value, this does not automatically translate to insight. Intelligent data, though useful, merely provides a more refined view of the raw information.
In the realm of big data, which encompasses vast datasets measured in gigabytes and exabytes, the process of adding intelligence through machine learning can yield actionable information. This includes annotating data with sentiment, emotions, and topics. However, while this can offer valuable information, it still falls short of the deeper, strategic understanding that constitutes genuine insight.
As we progress up the knowledge pyramid, it's clear that actionable information and true insight are often confused. While actionable information involves identifying and addressing specific issues—such as a customer pain point uncovered through various data sources like tweets or reviews—this type of information is typically used for immediate problem-solving and short-term improvements.
True insight, however, goes beyond merely reacting to identified issues. It involves a deeper understanding that informs long-term strategy and drives proactive decision-making. For instance, while solving a customer pain point is valuable, insight requires synthesizing data to predict future trends and shape strategic direction, offering a more profound and strategic advantage.
At the pinnacle of the knowledge pyramid, business insight is defined as a 'gold nugget' that emerges from synthesizing information across multiple sources and applying a measure of intuition. Unlike simple data points or actionable information, which can be derived from single sources or immediate problem-solving efforts, insight is strategic and requires a long-term approach to implement effectively. It involves a deeper level of understanding that leads to proactive decision-making and can drive substantial positive outcomes for a business.
This distinction highlights why insight is positioned above intelligent data and actionable information in the pyramid. While intelligent data and actionable information are crucial for addressing specific issues and reacting to opportunities, insight goes further by enabling predictive strategies and long-term success. Recognizing this hierarchy underscores the importance of not only collecting and analyzing data but also transforming it into meaningful insights, which remains a rare and highly valuable skill in today’s business landscape.
Insight, while emerging from data and actionable information, represents a higher echelon of strategic value. It transcends mere data analysis to offer predictive and proactive benefits, making it a rare and valuable asset at the peak of our pyramid. Understanding this distinction helps businesses leverage data effectively and grasp the true power of insightful analysis.
Original Source: Information Age
Artificial intelligence (AI) offers significant advantages across various industries, from healthcare and finance to retail and logistics. A key benefit of AI lies in its ability to enhance decision-making by analyzing user data through methods such as text and image analytics and emotion analysis. Michalis Michael, CEO of listening247, delves into how organizations can leverage AI to optimize user data analysis, thereby improving business decision-making and driving better outcomes.
Absolutely. In fact, we can use data – hard numbers like profitability, for example – to demonstrate that data driven businesses are often much more successful than those who do not use data for decision making. Working on the basis of ‘gut feelings’ may have a place in business management, but it’s no replacement for the insights that data provide.
If the right processes are in place: instantly. Of course, not all insights are created equal, so some of the more complex kinds of insight will require a human expert to identify them.
With that caveat aside, there are now sophisticated automation and sharing technologies which can handle data collection, cleaning, annotation, visualisation, analysis, and insight discovery. With the help of online dashboards and alerts, these processes enable data to be easily shared with an organisation’s management and implemented swiftly.
They certainly do. As I mentioned earlier, data is an increasingly essential tool for success in business, and the use of that data requires a certain amount of strategising. Businesses will need to decide, for example, what data to collect and how often. Data is all around us, so we need to be discerning about what to look for.
Not only do businesses need a strategy for what kind of data to collect (there is a huge variety of unstructured and alternative data to choose from, after all), but they will also need to decide which software tools and platforms are the most efficient and cost effective to collect the data, how they intend to add intelligence to the data, and – lest we forget – what kind of delivery mechanisms to employ.
In the trading and investment sector, ‘alternative’ data means any kind of data that falls outside the traditional fundamentals of a listed company. Examples of this standard kind of data include revenue, profit and price/earnings ratios.
Alternative data, by contrast, is any other kind of information. Social sentiment gleaned from Tweets, app usage, and even satellite images are forms of alternative data – and they contain the potential for insight and foresight in the right hands.
While many companies can benefit from harnessing alternative data, those operating in the financial services sector – quantitative funds, discretionary funds, private equity funds, and similar entities – tend to gravitate towards the world of alternative data to discover “new alpha”.
More broadly, alternative data is useful to corporates that want to predict sales or purchase intent. In those kinds of scenarios, companies can use KPIs from social intelligence. These can range from net sentiment scores to purchase intent posted on social media: alternative data sources that fall far beyond the interests of ‘traditional’ fundamentals, but which clearly have a huge bearing on companies interested in predicting the future stock price movements or actions of their prospective customers.
The longevity of relevance can vary enormously – it all depends on the sector involved and on the type of data we’re talking about. Suppose, for example, a company is interested in opinions expressed online about a big, long-standing topic like Brexit. In that case, I’d expect to see extremely slow depreciation as the issues rumble on. By contrast, gathering data on an advertising campaign sees very fast depreciation: in that scenario, the data is only as good as the most recent ad.
Sometimes, of course, the situation isn’t this clear-cut. Looking into social sentiment in order to predict stock price fluctuations, for example, can retain relevance for days or weeks depending on the ‘hype’ surrounding the stock. Data relevance certainly can depreciate very quickly indeed – that’s why, as far as tracking customer data is concerned, it’s advisable to implement tracking on a daily or weekly basis.
The analysis of unstructured data, including text, audio, images, and video, will soon become ubiquitous – engagement with this kind of data is swiftly changing from novelty to necessity. As such, our voices as customers – as users of products and services – will always be heard and responded to.
There are two positive implications for this. Firstly, we will be able to directly impact product development in direct, concrete ways that will improve human lives. Secondly, we’ll be able to monetise our own behavioural data and opinions. These changes in data analysis indicate a rise in the value of the data we put out into the world: it’s a precious resource and will likely be treated as such.
In conclusion, the strategic use of AI and data analytics is reshaping the landscape of business decision-making and customer experience management. The ability to harness comprehensive data insights—from traditional metrics to alternative data sources like social sentiment—empowers businesses to gain a competitive edge and respond swiftly to market demands. As technology evolves, the integration of advanced data analysis will become even more crucial, enabling real-time adjustments and enhancing product development. Companies that embrace these innovations will not only optimize their operations but also enhance customer satisfaction, making data a vital asset for future growth and success.
Original Source: Information Age
The exponential growth of Big Data is set to transform the insurance industry dramatically. According to Statista, global data consumption, which was 79 zettabytes in 2021, is projected to exceed 180 zettabytes by 2025. This surge in data will predominantly consist of unstructured data from various sources, including customer interactions and IoT devices. As insurance companies strive to enhance customer experience management (CXM) through comprehensive data analytics, they must adapt to manage and leverage this influx of information effectively. The integration of advanced technologies, such as edge computing and AI, will be crucial in handling the massive data volumes and improving forecasting accuracy, ultimately reshaping the industry's landscape. Our CEO, Michalis. A. Michael gives us a better understanding from his interview.
Globally, data consumption is expected to surge from 79 zettabytes in 2021 to over 180 zettabytes by 2025, with storage capacity growing at a 19.2% annual rate. Much of this data will be unstructured, stemming from customer interactions and IoT devices. This expansion offers a significant opportunity to enhance customer experience management (CXM) by incorporating comprehensive analytics across various communication channels and languages. Advanced AI can process and interpret this data, providing actionable insights regardless of language barriers.
IoT data – for example from wearable devices which measure biometrics or smart car devices that measure driving behaviour – will become standard for the insurance industry since they promote forecasting accuracy and competitiveness which is articulated in a given customer’s insurance premium.
The impact will be largely positive due to improved forecasting models. The data available currently and in the future is an actuary’s dream.
Rapid changes in data processing are anticipated in the coming years to handle the growing volume of data. According to Gartner, while only 10% of enterprise-generated data is processed outside traditional data centers or cloud environments today, this figure is expected to rise to 75% by 2025. This shift highlights that current cloud solutions will struggle with the vast amounts of data generated by IoT devices and other sources at the network edge.
Data centers alone will not be able to meet the demands for faster response times and higher transfer rates, leading to significant challenges for many applications. The solution lies in decentralization through edge computing, which brings data processing closer to the source. This approach will be crucial in managing the massive data consumption and ensuring efficient data handling and analysis.
Data privacy increasingly focuses on obtaining individual consent for personal data usage, and organizations are becoming more aware of information security. This heightened awareness extends to longer sales cycles for CXM programs due to stringent security protocols. Adhering to global standards like ISO27001 and ISO27002 and undergoing independent penetration tests helps identify and address vulnerabilities, thereby mitigating risks of data breaches.
The most significant shift is driven by AI's growing capability to accurately annotate and analyze unstructured data, which comprises over 95% of all data ever recorded. Unlike structured data, unstructured data includes text, audio, images, and video. This development has profoundly impacted customer experience management (CXM), enabling companies to access and understand every customer interaction, regardless of language, and effectively design and implement tailored customer service workflows and scripts.
As Big Data continues to proliferate, the insurance industry stands to benefit significantly from improved forecasting models and enhanced CXM capabilities. The rise of IoT data and unstructured information presents both challenges and opportunities, driving the need for innovative data processing technologies like edge computing. While the increasing volume of data raises concerns about privacy and security, adherence to global standards and rigorous security protocols can mitigate these risks. The future of Big Data in insurtech is poised for transformative change, with AI-driven analytics enabling insurers to gain deeper insights into customer interactions and refine their services accordingly.
Original Source: Global Banking & Finance Review
Michalis Michael, CEO of listening247, recently appeared on Kalkine Media Australia’s Expert Talks for a live interview focused on the role of AI in analyzing unstructured data. During the interview, Michalis delves into the significance of leveraging machine learning and AI to interpret vast and diverse data sets, including texts, audio, video, and images. He emphasizes the transformative power of these technologies in extracting valuable insights from unstructured data.
In his discussion, Michalis highlights how businesses can benefit from this advanced data analysis by enhancing their customer experience (CX). He explains how sentiment and topic annotation can provide companies with a clear understanding of their position in the customer journey and their relationship with their brand. This insight allows businesses to tailor their strategies and improve customer interactions.
For a deeper dive into Michalis Michael's insights on harnessing AI for unstructured data analysis, watch the full interview here: How to use AI to produce accurate, actionable, timely intelligence to unstructured data?
Original Source: Kalkline Media
Unstructured data (UD) is rapidly gaining prominence, with its importance set to soar throughout this decade. Recent reports highlight a surge in investment, with 45% of businesses prioritizing UD analytics software and 62.5% of IT leaders increasing their UD storage budgets by the end of 2021. Given that over 90% of global data is unstructured and IBM estimates a daily generation of 2.5 quintillion bytes, it’s clear that UD holds immense, untapped potential. However, the vast scale and complexity of UD necessitate specialized expertise to unlock its valuable insights and drive business growth.
Unstructured data (UD) includes diverse formats such as text, audio, images, and video, and stands in contrast to structured data (SD), which is organized into tables and easily searchable. While SD's organized nature allows for straightforward access and interpretation, UD's complexity arises from its lack of structure and its expression in various natural languages, making it challenging to process with traditional methods.
The sheer volume of UD generated daily—equivalent to ten million Blu-ray disks stacked as high as four Eiffel Towers—demonstrates its vast potential. However, unlocking this potential requires advanced AI technology and specialized expertise, highlighting the crucial role of companies that can effectively manage and analyze UD. This growing field, largely driven by start-ups and early-stage firms, offers significant business value and opportunities for innovation.
Businesses invest in unstructured data (UD) due to its unparalleled value in providing deep, actionable insights. In marketing intelligence, for example, AI can analyze vast amounts of customer interactions to reveal nuanced details about customer sentiment and experiences with a brand, service, or product—insights that are far more precise and actionable than traditional methods.
Additionally, UD plays a crucial role in trading and investment by analyzing news articles and headlines to identify factors influencing stock prices. However, the effectiveness of UD is contingent upon the accuracy of the analysis tools and human expertise involved. Poorly managed UD analysis can lead to costly errors, emphasizing the need for sophisticated technology and skilled professionals to extract meaningful insights and avoid misinformation.
The growing investment in unstructured data (UD) by IT leaders is no secret; it reflects the undeniable power of data in fields like marketing and alternative analytics. For investors aiming to harness this power, opportunities abound in supporting companies that excel at converting raw data into valuable insights. The true value of UD lies with the technology and AI experts who master its complexities, making this a promising area for investment.
Original Source: London Loves Tech
Research World recently highlighted the perspectives of Michalis Michael, CEO of listening247, in an in-depth article exploring the evolving landscape of social intelligence. His insights delve into the future of social listening and analytics, shedding light on emerging trends and innovations in the industry. This comprehensive piece is divided into two parts: "Remote Research: The Future of Asking Questions, Part 1" and "Remote Research: The Future of Asking Questions, Part 2," offering a thorough examination of these critical topics.
Research World, a leading market research association, is dedicated to advancing the field of market, opinion, and social research. By focusing on innovation and best practices, it provides valuable updates on the latest tools, technologies, and methodologies in data analytics. This publication helps professionals stay informed about the cutting-edge developments shaping the industry.
Original Source: Research World (part 1) & Research World (part 2)
Watch Webinar On-Demand: Click Here
Today, most brands are wanting to take increase opportunistic strategies in how they actively shape their brand's presence, especially when it comes to having a strong online brand presence. To truly understand and resonate with your audience, this session dived deep into helping brands understand their consumers behave online from recognizing their interactions, understand their needs, and delivering experiences that exceed expectations. With the help and organisation by our agency partner in Asia, this webinar was held On Tuesday June 6th at 9am UK time.
As brands want amplify their brand's presence and influence in the market across various social media platforms, social media listening has become of great importance in boosting your brand visibility and engagement, whilst dominating conversations and attract more customers online. This webinar offers a unique opportunity to find out why social listening is gaining importance so quickly in research, how good quality social listening is carried out and when social listening works for you and your clients.
Michalis is an expert in digital monitoring and social intelligence with extensive experience in transforming data into actionable business insights. Under his leadership, DMR has developed cutting-edge solutions like the Brand Health Tracking, which helps brands monitor their performance and reputation across various channels.
Phil expert in the use of MRDCL, the no.1 data tabulation and analysis software and interviewed Michalis, offering a unique opportunity to find out how Social Listening is changing the way many companies are spending their research budgets.
Don't let timing hold you back from transforming your brand into greatness! Watch the webinar now.
Watch Replay: Click Here
P.S. Have questions about the webinar? Contact us at: info@listening247.com. We're happy to help!
.jpg)
Watch Webinar On-Demand: Click Here
Brands are increasingly seeking innovative ways to engage with their audiences. communities247 stands at the forefront of this movement, offering a dynamic and fully customisable DIY Online Communities platform designed for consumer engagement, brand advocacy, insights, and co-creation. With the help and orrganisation by our agency partner in Asia, MRDC Software, This webinar was held On Wednesday August 23rd at 9am UK time.
As brands strive for novel methods to connect with their audiences, communities247 emerges as a pioneering force, providing a vibrant and entirely adaptable DIY Online Communities platform crafted for fostering consumer engagement, brand advocacy, insights generation, and collaborative creation. This webinar offers a unique opportunity to find out how to make the most of Private Online Communities to reach consumer insights on demand.
Michalis is an expert in digital monitoring and social intelligence with extensive experience in transforming data into actionable business insights. Under his leadership, listening247 has developed cutting-edge solutions like the Brand Health Tracking, which helps brands monitor their performance and reputation across various channels.
Don't let timing hold you back from transforming your brand into greatness! Watch the webinar now.
Watch Replay: Click Here
P.S. Have questions about the webinar? Contact us at: info@listening247.com. We're happy to help!
Watch Webinar On-Demand: Click Here
We are excited to invite you to an exclusive webinar hosted by Michalis A. Michael, CEO of listening247, where he will introduce DataVinci, our advanced Generative AI (Gen AI). This event is designed for consumer brands interested in using data and AI to enhance their marketing strategy and make smarter, more informed decisions.
The webinar took place on Wednesday, 30th of October at 1:30 PM (UK time), and last s45 minutes, including a Q&A session and interactive polls. This online event will provide valuable insights on how brands can better understand their customers, improve engagement, and optimise their marketing efforts.
But that’s not all! We’re also excited to introduce our special guest speaker, Tom Molnar, Co-Founder and CEO of GAIL’s, the renowned British bakery and coffee shop chain.
During the webinar, Michalis A. Michael will explain how DataVinci combined with listening247’s Social Listening and Analytics, can change the way you use digital media.
Tom Molnar, CEO of GAIL’s, will provide a real-world look at how DataVinci is impacting GAIL's business.
Michalis will explain how listening247’s Social Listening and Analytics solution gathers data from platforms like Instagram, X, TikTok, Facebook, and more. It also tracks mentions in news articles, forums, and online publications, providing a full view of your brand’s digital presence.
With this information, listening247 tracks and analyses key factors such as brand sentiment and customer opinions.
DataVinci then takes this data and turns it into meaningful insights. Using advanced AI capabilities, it identifies patterns and opportunities within the data and translates these findings into detailed reports and strategic recommendations. These reports don’t just highlight trends; they also provide the major conversation drivers that influence how your brand is perceived as well as specific actions that your marketing team can take to improve engagement and align with customer interests.
But DataVinci doesn’t stop at analysis. It goes further by offering ready-to-go social media posts, complete with image briefs, designed for your brand to publish. This feature simplifies the content creation process and ensures that the messaging is data-driven and resonates with your audience. DataVinci also helps your team stay on top of trends and topics that matter to your customers, ensuring that your content remains relevant and impactful.
This webinar is tailored for C-Suite Execs of SMEs consumer brand leaders, marketers, and decision-makers who want to use data more effectively. It’s an opportunity to learn how Gen AI can help you make informed decisions, improve customer engagement, and stay competitive in today’s fast-paced digital landscape.
Watch Replay: Click Here
P.S. Have questions about the webinar? Contact us at: info@listening247.com. We're happy to help!
On November 8, 2023, Michalis Michael, CEO of listening247 and GIHE Research Fellow, spoke at the Montreux International Tourism Forum in Switzerland. He discussed the role of AI in analyzing unstructured data within the tourism industry. You can watch his full presentation here.
1. Human resources: what are the solutions to the current shortage?
2. Artificial intelligence and tourism – definitions and realities;
3. Unlocking the potential of AI for tourism SMEs: insights from Europe-wide research;
4. “Service recovery” in tourism in a world of multiple crises;
5. Tourism and sustainable development;
6. A life for tourism.
1. Any researchers and any professional involved in the toursim sector.
2. Organisations that prioritise toursim strategies and operations that involves some form of A.I.
P.S. Have questions about the ebook? Contact us at: info@listening247.com. We're happy to help!
On October 10, 2023, Michalis Michael, CEO of listening247 spoke at the Data & Insights Network’s online event, "AI for Marketing & Business." His presentation was presented in English. For more details, visit the event page, which can be translated using Google Translate if needed.
1. Gain a deeper understanding of how AI is transforming marketing and business practices;
2. Delve deeper into the opportunities and threats of generative AI,
3. Engage with fellow members and industry experts in a dynamic, digital setting;
4. Understand the potential challenges and ethical considerations of using generative AI in business;
5. Experience an online Education Day tailored specifically for members, providing an intimate and focused learning environment.
1. This session (entirely in English) is exclusively for (teacher) members of the Data & Insights Network and NIMA
2. Anyone who is interested in becoming a member, and is interested in the education sector.
3. All those who play a crucuial role in the education sector.
P.S. Have questions about the ebook? Contact us at: info@listening247.com. We're happy to help!
This material includes a used case infographic, which presents the 20 Questions you should be asking when evaluating Social Media Listening vendors. It dives deeper into helping organisations understand the process in choosing the right tool for their organisation and whatever they are trying to achieve.
P.S. Have questions about the ebook? Contact us at: info@listening247.com. We're happy to help!
.jpg)
This material includes a used case infographic, which demonstrates how text analytics alone can miss key insights as more people use images and videos to express their views on social media. To fully understand brand sentiment, a deep learning approach can be employed to analyze visual content, while speech-to-text technology converts audio into text for comprehensive analysis
P.S. Have questions about the ebook? Contact us at: info@listening247.com. We're happy to help!
This ebook explores the evolving discipline of social listening, its many names, and the key questions organizations should ask before getting started. If you are new to social media monitoring or don’t have the answers to these questions, read on...
1. See how web listening provides a unique consumer portal at a time when response rates for research projects are dropping and it’s getting harder to reach your key customers;
2. Understand why It is now more vital than ever for organizations to understand what is being said about them, so that they can effectively manage their brands and their reputation online;
3. Learn how findings from web listening can improve business performance, whether it’s communication, targeting or product development or using the findings to enhance data from existing sources.
1. Any organisation, brand manager and employees within the marketing department who specialise in the discipline of social media.
2. Market research agencies who want to learn about social media monitoring or web listening.
P.S. Have questions about the ebook? Contact us at: info@listening247.com. We're happy to help!
The future of Customer Experience Management (CXM) is set to be transformed by the ability to analyze all customer interactions, regardless of their source or language. This shift is not speculative but rather a natural progression of existing technology. With AI-driven tools becoming increasingly capable of processing vast amounts of customer data, the industry is moving towards a universal approach that captures every customer opinion, whether solicited or unsolicited. As CXM professionals adapt to this new reality, they will face both challenges and opportunities in harnessing the full potential of this comprehensive analysis.
The future of Customer Experience Management (CXM) will be significantly impacted by the ability to access and analyze all customer opinions, particularly in tracking multiple relevant KPIs. While many companies currently rely on the Net Promoter Score (NPS) to gauge customer loyalty and satisfaction, this single KPI may not capture the full complexity and nuance that new technology offers by analyzing every customer interaction.
As CXM evolves, we can expect a shift towards using multiple, composite KPIs that synthesize data from various customer interaction sources, such as sentiment, purchase intent, and social media engagement. While these findings may still be presented in a simplified score, like DMR’s Net Sentiment Score (NSS) or Social Presence Score (SPS), the measurement of market performance will need to reflect the sophistication and depth of the available data.
Access to comprehensive customer interaction data will not only enhance market position insights but also revolutionize how companies address pain points. With all customer feedback at their disposal, businesses will be able to prioritize and address issues more effectively, both proactively and reactively. Reactive measures may involve refining call center protocols and complaints handling, while proactive solutions could focus on real-time operational fixes to mitigate issues before they escalate.
These advancements are already feasible with current technology and specialist expertise, enabling companies to convert customer opinions into actionable strategies. By tackling pain points and optimizing customer retention, businesses can significantly reduce churn. Future discussions will shift towards leveraging customer strengths and gain points to further improve CX.
The growing reliance on advanced technology for Customer Experience Management (CXM) will introduce new challenges, particularly regarding the accuracy of AI-driven annotations for sentiment and purchase intent. Ensuring precise analysis of millions of interactions across various languages is crucial for effective CXM. As the industry adapts to these technologies, a virtuous cycle may emerge: enhanced annotation accuracy leads to better analysis, which in turn improves CXM capabilities, promising robust business growth and a more satisfying customer experience.
In conclusion, the integration of advanced AI technologies in Customer Experience Management (CXM) holds the promise of transforming how businesses understand and respond to customer interactions. By leveraging precise, language-agnostic data analysis, companies can move beyond traditional KPIs and gain a deeper, more nuanced view of customer sentiment and behavior. This shift enables proactive and reactive solutions to pain points, ultimately enhancing customer satisfaction and retention. As the industry embraces these innovations, the emphasis on accurate data annotation and comprehensive analysis will drive significant growth and elevate the overall customer experience.
Origal Source: Finance Digest
This ebook aims to demonstrate how integrating Social Listening and Analytics data with survey data, and other traditional research methods, can help companies uncover some of the most accurate insights possible. A case study revolving around Health & Wellness, as discussed online in Arabic, was selected as an example to illustrate the untapped power of social listening analytics.
1. Find out how to uncover and synthesize accurate insights by integrating data from various sources;
2. Discover the untapped power of social listening analytics;
3. Understand why traditional methods are no longer sufficient or fully representative;
4. Learn about trends and ideas related to Health & Wellness in the MENA region.
1. Any organisation within the health & wellness sector who want to enhance their market research activities to uncover some of the most accurate insights possible.
2. Market research agencies who want to learn about the importance of accuracy when it comes to social insights about trends and ideas related to health and wellness in the MENA region.
P.S. Have questions about the ebook? Contact us at: info@listening247.com. We're happy to help!
