Artificial Intelligence for Data Analysis
What is it, how it works, & what you need to know
Artificial intelligence (AI) is the capability of a machine to imitate intelligent human behavior.
Computers have long been able to remember information, and execute simple logic statements. Artificial Intelligence still uses both of these basic capabilities, but on a much, much larger scale.
There have been major advances in processing power and storage, not to mention cloud computing. Thanks to these, we’re now able to build machines that come close to mimicking human intelligence. Although the exact ways humans and machines learn and reason still differ on a technical level, there are many points where they overlap.
We hear about lots of AI examples you hear about today – from gameshow winning computers to self-driving cars. They all rely on techniques like deep learning and natural language processing.
Computers can be trained to accomplish specific tasks by processing large amounts of data and recognizing patterns.
The History of Artificial Intelligence
It’s not as new as you think
The term artificial intelligence was first used back in 1956. AI has become much more popular today thanks due to its promising capabilities and anticipated benefits.
Early AI research in the 1950s explored topics like problem solving and symbolic methods. In the 1960s, the US Department of Defense took interest in this type of work and began training computers to mimic basic human reasoning. For example, the Defense Advanced Research Projects Agency (DARPA) completed street mapping projects in the 1970s. DARPA successfully built intelligent computer assistants in 2003, long before Siri or Alexa.
This early work paved the way for the automation and formal reasoning that we see in computers today, including decision support systems and smart search systems that can be designed to complement and augment human abilities.
While Hollywood movies and science fiction novels depict AI as human-like machines with a knack for harming their creator, current AI technology isn’t that scary – or quite that advanced either. Instead, AI continues evolving and promising many specific benefits in every industry.
1950s–1970s: Neural Networks
Early work with neural networks stirs excitement for “thinking machines.”
1980s–2010s: Machine Learning
Machine learning becomes popular.
Present Day: Deep Learning
Deep learning breakthroughs drive AI boom.
AI has been an integral part of Inzata for years. Today we help customers in every industry capitalize on advancements in AI, and we’ll continue embedding AI technologies like machine learning and deep learning in solutions across the Inzata portfolio.
Artificial Intelligence and Machine Learning
They’re similar, but not identical
These two terms are often used together, sometimes interchangeably. Machine learning is actually a branch of AI that deals with how machines can learn from data from earlier events.
Specifically, machine learning involves using their capacity to ingest huge quantities of data in order to form conclusions. You may recall the Hollywood trope of a robot character flipping through the pages of hundreds of books at superhuman speed. The robot then finishes, looks up, and announces something intelligent, or gives some conclusion. That film Depicition is not too far off from how it really works.
Software algorithms start by reading millions of earlier events, called “features”. They then start scoring patterns among the features based on how the observed values interact with one another.
For example, lets say you wanted to train an algorithm that predicts whether it’s raining on a given day. You could do so by showing it photos from the same city street on thousands of different days and let it count the number of people with umbrellas or raincoats. Eventually the algorithm would figure out that the more umbrellas it sees, the more likely it was raining that day.
Why is Artificial Intelligence Important?
What can it really do?
AI automates repetitive learning and discovery through data.
But AI is different from hardware- and sensor- driven, robotic, “If this then that” automation. First, it appears to be more nuanced, which mimics human intelligence. This is because the AI is considering hundreds if not thousands of different factors in its decision making, not just the most obvious ones. But this does not make it human, it merely means that a human programmed it with some very thoughtful and exhaustive logic.
A great example is lawn sprinklers timers, and how they can make smarter decisions the more data they’re given:
- Their basic logic starts with a timer, “Water every other day at 6 am.” This is a pretty rigid rule.
- The “smart” system I have now connects to the Internet. Via a mobile app, it’s able to access today’s weather forecast and override the scheduled watering if rain is in the forecast.
But just because rain is forecast doesn’t mean it will rain exactly on my lawn, (weather is just like that.)
Deluxe models can add a soil moisture sensor into the decision circuit that detects how wet my lawn is and decide if it needs watering.
So now with just 3 datapoints to go off of, my lawn sprinkers have gone from useful but dumb to pretty much as smart as a human (at least when it comes to lawn watering.) Keep in mind that mature AI algorithms can process thousands of individual data points per second.
Adaptive AI adapts through progressive learning algorithms to let the data do the programming
This involves giving the AI some free reign to begin adapting it intelligence to its environment. AI finds structure and regularities in data so that the algorithm acquires a skill: The algorithm becomes a classifier or a predictor. So, just as the algorithm can teach itself how to play chess through trial and error, it can teach itself what product to recommend next online. If you’ve ever observed a toddler learning how to walk, you notice they fall down a lot. You could say the toddler is really mastering not falling down before learning to walk. Learning algorithms are the same way. As long as the AI can tell the difference between a good and bad outcome, it will keep trying until it achieves the good outcome most of the time. That’s why models constantly need new data to adapt. Back propagation is an AI technique that allows the model to adjust, through training and added data, when the first answer is not quite right.
AI adds intelligence to existing products
In most cases, you will never see a product called “AI.” Rather, products you already use will be improved with AI capabilities. My “dumb” irrigation timer matured into a smart one. Siri, or how Siri was added as a feature across the entire line of Apple products.
AI is being taught to analyze more and deeper data using neural networks that have many hidden layers
Building a system with five hidden layers was almost impossible a few years ago. All that has changed with cloud computing power and an ever increasing volume of data. You need lots of data and speed to train deep learning models because they learn directly from the data. The more data you can feed them, the more accurate they become.
AI achieves incredible accuracy through deep neural networks – which was previously impossible
For example, your interactions with Alexa, Google Search and Google Photos are all based on deep learning – and they keep getting more accurate the more we use them. In the medical field, AI techniques from deep learning, image classification and object recognition can now be used to find cancer on MRIs with the same accuracy as highly trained radiologists.
AI improves the ROI of most data
With AI’s ability to learn from even the most mundane data, previously undervalued data sources are experiencing a renewed appreciation. When algorithms are capable of teaching themselves, self-learning, the data itself becomes valuable intellectual property. The answers are in the data; you just have to apply AI to get them out. Since the role of the data is now more important than ever before, it can create a competitive advantage. If you have the best data in a competitive industry, even if everyone is applying similar techniques, the best data will win.
How Artificial Intelligence is Being Used
Examples of AI use cases
AI Can Deal With the Data Deluge
The sheer volume and variety of data coming at companies is a growing problem in the BI space.
With more data sources than ever before to choose from, A medium sized company often has something as critical as customer data can be stored across 10-15 different systems. CRM, ERP, Marketing, Support, all contain unique attributes about customers and their activity.
There are many BI tools focused on the analysis of data. The issues with these is when the volume and variety of data grows. BI tools are great for pulling and analyzing data from 1-2 of these.
While most BI solutions can process and store a huge amount of data with many dimensions, they don’t offer an easy way to get insights from the data. To find ways for the business to improve its key KPIs, data analysts simply don’t have the capacity to keep up with the increasing demand to crunch all the data. In fact, BI solutions have largely left the “I” – the intelligence – completely in the hands and minds of the data analysts. The human brain is limited in the number of data points it can process and correlate.
According to Gartner, Inc., “More than 40 percent of data science tasks will be automated by 2020, resulting in increased productivity and broader usage of data and analytics by citizen data scientists.”
AI stands to play a greater role in BI, where intelligent systems pour over more data than any human could reasonably examine. “With millions of metrics coming in daily, companies don’t have the ability to efficiently track vast amounts of customer data without risking the potential of missing essential insights, which leads to damage monetarily and reputationally,” said David Drai, CEO and Co-founder of Anodot. The more data analysts can identify good and bad deviations from the norm, the more quickly they can react to changes in the business and take necessary action.
New Tools, Same Disruptions
AI analysis is not unlike previous technological disruptions; the printing press made calligraphers obsolete, but introduced the new role of the professional printer. While AI analysis stands to disrupt BI, it opens the door for new jobs.
David Crawford writes in VentureBeat, “The work of an analyst, however, does not just involve conducting data analysis within closed environments. The analysis must be applied to the outside world where there is much more context influencing the interpretation. For example, while AI connected to sensors might be able to analyze the soil on a plot of land and optimize yield more efficiently than a human, it doesn’t know what impact the soil conditions have on the flavor of the resulting crop.”
Going forwards, AI will help provide focused insights for data analysts, by reading more deeply into data and identifying patterns. Carrying out exploratory tasks, such as recognizing specific deficiencies or untapped opportunities among the data, will help human professionals to interpret these discoveries to make more informed decisions.
The Value of Data Analytics is Growing
Big Data thought leader, Bernard Marr adds, “As the value of data analytics becomes apparent in all fields of activity, a growing number of people will want to be able to extract insights from their data. They might not want to take three or four years out to learn advanced computer science and statistics, and with the advances in cognitive computing that won’t be necessary. All that is required might be a brief introduction to NLP technologies.”
Joel Shapiro, executive director of the data analytics program at Northwestern University’s Kellogg School of Management says, “Analytics still rests fundamentally on good critical thinking skills —how to ask good questions and rigorously assess evidence that can lead to action.”
Artificial intelligence addresses today’s data deluge better than humans, since human analyst can’t sift through all of this data unaided. You can’t have a person sitting there or even whole teams watching dashboards to protect a brand or expect them to zero in on business incidents as they happen. You need AI tools.
AI Enhances Data Analyst Job Security
This doesn’t mean AI is coming to eliminate jobs for those involved in BI. While AI can do the work that no one has the time for, companies will come to see much stronger benefits in BI and be more inclined to further invest time and effort — creating more jobs in the field as a result. AI is good for job security.
AI and data analytics were developed by humans, for our own benefit. David Crawford adds that, “Understanding what it means to be human and caring about the human experience are intrinsically related to the analysis process.” Human data analysts aren’t going away as long as other humans remain their ultimate consumers. Data analysts will become ‘managers’ of teams of AI ‘employees’, leveraging the AI’s algorithms to comb through data and to even get answer to questions that weren’t asked.
As these systems collect and interpret greater volumes of data than we ever could, they advance, learning from past analyses to see what’s worked well. As David Drai observed in VentureBeat, “All advances in A.I. are built on the premise that if we can teach machines to learn from their “experiences,” then they will be able to more effectively sort through new information and help us flag the pieces that we need to know about immediately. Obvious steps forward, like the capacity to more effectively recognize seasonality or expect “unexpecteds,” will help lower the number of false positives and enable a far greater reliance on BI.”
These systems still require a human to design and maintain them, to ask them the most important questions for the business, and to communicate their results with colleagues in other specialties.
As AI solutions are able to dig deeper and more quickly link a cause with an effect, they can drastically reduce the time it takes to prevent or handle a crisis. This empowers the business, uncovering unforeseen opportunities while creating new means of driving revenue and enabling far more insightful decisions for data analysts.
With highly scalable machine learning-based algorithms, we now have software that can learn the normal pattern of any number of data points and correlate different signals to accurately identify anomalies that require action or investigation – by the data analysts.
Data analytics is no fad. In fact, the global market for data analytics has been predicted to exhibit a CAGR of 30.08% between 2017–2023 to surpass a valuation of USD 77.64 billion. A large part of this is due to the increased generation of data during the period but far more because of the increasing ability to use statistical algorithms and machine learning techniques to deliver actionable results for businesses.
“People used to say that information is power but that is no longer the case. It’s the analysis of the data, use of the data, digging into it — that is the power”
From a business perspective, data analytics can be used to increase revenue, respond to emerging trends, improve operational efficiency and optimize marketing to create a competitive advantage. However, with so many buzzwords flying about such as data lakes, machine learning and artificial intelligence, it can be difficult to understand where the value is coming from and what an external provider can offer.
Structuring the Data
One of the most difficult challenges faced by organizations in the field of analytics is that data sources have historically been very difficult to analyze. As data sources are often disparate and fragmented, there has been a requirement for manual data cleansing prior to analysis. Studies show that this process of data preparation takes around 80% of the average analysts time.
In addition to this, much of the information generated by businesses has little or no formal structure; contracts, surveys and emails all hold a wealth of knowledge that analysts could use to uncover opportunities.
Trending AI Articles:
This work has often involved use of external consultants or significant investment in employee time. As a result, businesses require what we’d call an ‘opportunity cost’ and this is often restrictive or prohibitive in the adoption of data analytics or business intelligence platforms. That’s where text analytics comes in.
With the advent of machine learning, text analytics has advanced to a level where it is capable of exploring large numbers of interrelated features, bringing structure and clarity to documents and data. Taking invoices as an example, companies such as Rossum.ai are able to remove the need for manual processing and extract the key information into a structured table. But consider applying similar techniques to contracts, spend data and other usage data and it becomes clear that there could be a wealth of knowledge in analyzing these datasets in combination; this is what VisionClerk do.
Performing the Analysis
When it comes to analytics Deep Learning is often raised a potential solution to automatically extract meaningful patterns from large datasets for decision making. However, the key here is truly defining and understanding the goals of your analysis. Pre-prescribed rules with specific logic and decisions are still invaluable in helping users uncover meaningful opportunities with a full understanding of where the information is coming from.That’s where partnering with an organization that focus specifically on the analysis you’re looking to perform can be advantageous. Businesses often face a dilemma between brining in additional employees or forming links with external partners; the latter becoming far more attractive with the relative scale enabled by cloud platforms.
Linking your data with companies who specialize in a singular pursuit and direct focus on the problem you are trying to solve can ensure that you get consistent insights into the most relevant opportunities for your business. This collaboration can help uncover unique perspectives that working by yourself never could, and expand your thinking beyond what you realized was possible.
AI in a Variety of Industries
The possibilities are endless
Every industry has a high demand for AI capabilities – especially question answering systems that can be used for legal assistance, patent searches, risk notification and medical research. Other uses of AI include:
AI applications can provide personalized medicine and X-ray readings. Personal health care assistants can act as life coaches, reminding you to take your pills, exercise or eat healthier.
Artificial Intelligence enhances the speed, precision and effectiveness of human efforts. In financial institutions, AI techniques can be used to identify which transactions are likely to be fraudulent, adopt fast and accurate credit scoring, as well as automate manually intense data management tasks.
AI provides virtual shopping capabilities that offer personalized recommendations and discuss purchase options with the consumer. Stock management and site layout technologies will also be improved with AI.
AI can analyze factory IoT data as it streams from connected equipment to forecast expected load and demand using recurrent networks, a specific type of deep learning network used with sequence data.
Working Together with AI
Humans and robots…what could go wrong?
Should data analysts and other “knowledge sector” employees feel threatened by AI? According to many prominent experts observing the AI industry, there’s no need to worry. While AI will indeed bring significant changes, AI advances will continue to require human attention to ultimately make efficient and productive decisions.
Artificial intelligence is not here to replace us. It’s here to take away the drudgery of boring tasks do more of what we enjoy, just like any other mechanical creation. It augments our abilities and makes us better at what we do. AI algorithms learn differently than humans, and have certain strengths apart from humans. They look at things differently. They can see relationships and patterns that escape us. On the flip side, there will always be things humans are better at and can help AI with. This alliance of human and artificial intelligence offers many opportunities:
- Bring analytics to industries and aid decision-making where data is currently underutilized, especially down at the line-operation level.
- Improve the performance of existing analytic technologies, like computer vision and time series analysis.
- Avoid potential errors in decision-making and judgement and ensure consistent quality across operations.
- Break down economic, language and translation barriers, allowing people to collaborate across cultures.
- Augment existing abilities and make us better at what we do.
- Give us better vision, better understanding, better memory and much more.
What are the challenges of using AI?
It’s not always smooth sailing
Artificial intelligence is going to change every industry, but we have to understand its limits.
One thing to understand about AI is that you need good quality data to end up with quality AI. AI learns from the data, so if you feed it poor data, it’s going to show in the results.
The other double edged sword of AI is its speed. Just as it can learn and decide quickly, it can also make and multiply mistakes quickly. It is vitally important to test and monitor AI to avoid any unwanted results. Care must be used before adding layers of prediction or analysis to ensure they’re built on a solid foundation.
Today’s AI systems are trained to do a defined, specific task. The system that plays poker cannot play solitaire or chess. The system that detects fraud cannot drive a car or give you legal advice. So think of AI as specialty uni-taskers, at least for the moment.
In other words, these systems are very, very specialized. They are focused on a single task and are far from behaving like humans, who are natural multi-taskers.
Likewise, self-learning systems are not autonomous systems. The imagined AI technologies that you see in movies and TV are still science fiction. But computers that can probe complex data to learn and perfect specific tasks are becoming quite common.
AI-powered Data Mining and Machine Learning
AI is simplified when you can prepare data for analysis, develop models with modern machine-learning algorithms and integrate text analytics all in one product. Plus, you can code projects that combine SAS with other languages, including Python, R, or Java.
How Artificial Intelligence Works
AI works by combining large amounts of data with fast, iterative processing and intelligent algorithms, allowing the software to learn automatically from patterns or features in the data. AI is a broad field of study that includes many theories, methods and technologies, as well as the following major subfields:
- Machine learning automates analytical model building that evolves by learning from big data. It uses methods ranging from neural networks, statistics, operations research and physics to find hidden insights in data. What’s unique about ML is that the program is not told explicitly what to look for in the data. It uses math and statistics to determine which results qualify as a “good insight.”
- A neural network is a type of machine learning that is made up of interconnected units (like neurons). They take external inputs, and and process them through a multi-step network. The process requires multiple passes at the data to find connections and derive meaning from undefined data.
- Deep learning uses huge neural networks with many layers of processing units. With advances in computing power and improved training techniques to learn complex patterns in large amounts of data. Common applications include image and speech recognition.
- Cognitive computing is a subfield of AI that strives for a natural, human-like interaction with machines. The goal of cognitive computing is to simulate human thought processes in a computerized model. An example would be the ability to interpret speech, and carry on a normal conversation with a human. Chatbots that mimic human interactions are another.
- Computer vision relies on pattern recognition and deep learning to recognize what’s in a picture or video. When machines can process, analyze and understand images, they can capture images or videos in real time and interpret their surroundings.
- Natural language processing (NLP) is the ability of computers to analyze, understand and generate human language, including speech. The next stage of NLP is natural language interaction, which allows humans to communicate with computers using normal, everyday language to perform tasks.
Additionally, several technologies enable and support AI:
- Graphical processing units are key to AI because they provide the heavy compute power that’s required for iterative processing. Training neural networks requires big data plus compute power.
- The Internet of Things generates massive amounts of data from connected devices, most of it unanalyzed. Automating models with AI will allow us to use more of it.
- Advanced algorithms are being developed and combined in new ways to analyze more data faster and at multiple levels. This intelligent processing is key to identifying and predicting rare events, understanding complex systems and optimizing unique scenarios.
- APIs, or application processing interfaces, are portable packages of code that make it possible to add AI functionality to existing products and software packages. They can add image recognition capabilities to home security systems and Q&A capabilities that describe data, create captions and headlines, or call out interesting patterns and insights in data.
In summary, the goal of AI is to provide software that can reason on input and explain on output. AI will provide human-like interactions with software and offer decision support for specific tasks, but it’s not a replacement for humans – and won’t be anytime soon.