Don’t believe the hype, there’s a new headline every day talking about Artificial Intelligence (AI) changing this or that industry. People get hung up on the idea that AI will solve all their problems and therefore want to jump head first into starting an AI project.

With that mentality you’ll set yourself up to fail. A lot of people do. Most Machine Learning or AI consultancies get stuck building Proof of Concept (POC) one after the other with no progression to an applied AI solution across the industry. Enterprises are silo-ing their efforts, with minimal budgets or oversight to a complete AI roadmap and so these bitty AI projects end up abandoned.

You need clear planning across the whole organisation, buy in from every department, and a clear understanding of the limitations.

The main two I’ll talk about today are People and Data.

People

The issue here is multi-layered:

Do you have the right people to run and manage an AI project?

Do the people in the organisation have the right mindset, processes and authority to make an AI project a success?

You don’t have to be an AI expert to successfully run an AI project. You need to be someone that can understand the limitations and constraints both on a technical side, and the business side.

Being able to win favour with various departments, not least IT, is extremely valuable. As they will fundamentally be the controllers over the systems, data and resource that is needed. Plus you’ll need to get multiple department heads to sign off on budget. As more often than not the AI solution, for example a Content Recommendation Engine, will require both marketing, customer, and IT teams to be involved.

On a technical side, you don’t need to know the intricacies of an algorithm that will be used. But you need to understand the principles behind it. Take a Supervised Machine Learning model. What amount of training is required? You’ll most likely need domain experts to train the system, i.e. an administrator, or content curator.

Be clear with all stakeholders what it’s going to take to make an AI project a success from the out-set, and get everyone bought in with clear costs and timelines set against a Return On Investment.

Finally, regarding processes. This ties to the data limitations that I’ll mention shortly. However the process of manually performing a task can be so valuable to the design and build of an AI solution. Let’s take Document Classification for use in an Incident Management system. Customer Support Level 1, writes a ticket. Typically includes a brief description of the problem. If that L1 support ticket doesn’t include enough detail, or is miss representing the real problem in some way, then when it comes to using those tickets for training a classifier, the Data Scientist involved will have to do an awful lot of data cleaning. It also makes the data less reliable, affecting the accuracy of the classifier.

If you ensure your teams follow correct procedure, and the data capture is accurate, you will save yourself an awful lot of time and money when it comes to building an AI solution.

Data

Clive Humby famously coined the phrase “Data is the new oil”, it sure is. But only if you know what to do with it. Data can be useless unless it’s accessed, stored and processed properly.

Most people only think of data as being a row of text or a number in a database. However we can break data down into two types, structured and unstructured, the former being your date of birth in a database at your doctors.

Unstructured data can be things like webpages, images or voice recordings. These are all made up of bytes, but a machine can’t understand the meaning of the data until its given structure in a database.

Most businesses today have over 80% of their data as unstructured. Just think, all your emails, your phone calls, invoices, the powerpoint presentation you worked on yesterday. It’s all unstructured. But, there’s so much value within.

This data can also be internal or external. With external data including things like Tweets, Market Reports, or Stock information.

When thinking about AI solutions for your project, you need to think about these types of data, as the harder to reach data will create additional degrees of complexity. What type of data are you building your AI solution with?

An example of a Structured Data solution is Churn Prediction. Let’s pretend you have a mobile app, there’s hundreds of thousands of users that have interactions on your app daily. They interact through ‘open’, ‘close’, ‘like’, ‘comment’ and ‘send’ interactions. These are structured in a database, i.e. Firebase. With those sequences of events, you can build a Machine Learning based Churn Prediction model that maps the behaviours to those likely to leave (churn) from your app.

No worries then, the data is stored in a database, it’s clean, and you understand it. That makes a Data Scientists life a lot easier when embarking on a project. You’ve got a higher chance of success.

Dealing with the 80% of unstructured data in an organisation is a different ball game all together. If you want to use that data set to build AI solutions then you need to think about the AI Creation Hierarchy Of Needs:

As illustrated here, you can see the steps to go through before getting your unstructured data into a position where AI or Deep Learning can take place. It’s a slow process, that includes building the foundations for data storage, using Data Warehousing solutions, like Amazon’s Redshift to handle these vast amounts of data. There’s an awful lot to be said here, and not enough for this post. But without the right building blocks for your data, then you can never hope to have a successful AI project.

This can really pull on resource requirements from IT to get a Data Warehouse built. But in the long-term it’s going to save more time for a higher cost Data Scientists or Machine Learning Engineer. Plus it will provide you with the building blocks for an Enterprise wide AI, that could lead to the development of further solutions.

Conclusion

There are so many more facets to consider when embarking on an AI project. Data and People are the two that if thought about correctly will give you the highest chance of success.

I’ve not touched on ethics or change management either. But you can imagine the impact a new AI project might have on the People who haven’t been properly educated on its benefits.

Most of the solutions we design have a human in the loop, which ensures not only a higher level of control and assurance, but also acceptance.

Don’t be blindsided by thinking AI will solve all your problems and get disappointed when it doesn’t. Take baby steps and plan properly, or talk to an expert. Ensuring you have all the right pieces in place before embarking on an AI project will lead to greater success.

About the Author

Jack Hampson is the CEO and Founder of Skim Technologies, a boutique Data Science and Machine Learning Consultancy that combines deep subject matter expertise and a set of proprietary technology accelerators to help make its enterprise and scale-up customers more successful.