How is Cognitive Computing Different from Big Data and NLP
As a business we often walk away from situations that can be solved using traditional data science and big data technology. This is purely a business decision – structured data science market already has eminent players, using Coseer or Watson for these problems will be an overkill, and we don’t want to reinvent the wheel. This is not a technology decision. Truth is data science used for cognitive computing is far more advanced than for big data problems.
Cognitive computing simply means teaching computers how to think and process information like humans. One of the key applications of cognitive computing is natural language. Again, its a business decision – being able to process natural language without any human intervention is a huge problem today. As computers get smarter, they will take more and more decisions in the enterprise world, all based on natural or unstructured inputs. In our day to day lives as well natural language interaction with our machines will become the pervasive interface. With Droid, Siri, Cortana, etc. the building blocks are already in place.
Truth is data science used for cognitive computing is far more advanced than for big data problems.
Still, natural language is only one application for cognitive computing. Advanced cognitive systems are already being developed that relate with vision, speech, etc. Cognitive computing could also be a great tool to solve the traditional big data problems, esp. where data size results in prohibitive compute cost or latency; or where underlying data is too dynamic; or when structured data is only one piece of a larger puzzle.
Here are the advantages of cognitive computing vs. traditional big data technologies for these problems:
- Scalability: Cognitive computing is all about forming hypotheses, proving or disproving them, and learning from them to form new hypotheses – essentially how humans think. Congnitive systems have memory, they have the ability to second guess themselves, and they are designed to go back and forth for finding the right answers. It means they can come up with equally rigorous insights without having to brute force their way through entire databases. They make fewer redundant calculations, a hallmark of scalability.
- Dynamism: The same capabilities give cognitive comptuers a lot of flexibility in ingestion and processing. If, for example, a system is processing some data batches, and a new variable shows up in a batch, cognitive computers will not skip a beat. They will automatically change models to include the new variables. For traditional technologies models may have to be rewritten and the performance will suffer by that much. This also holds true for situations where the content itself keeps evolving.
- Natural Interaction: Given that natural language is a prominent application in any case, pulling in natural language capabilities makes cognitive computing systems very powerful. For example, these systems can output their insights directly in business language. More importantly, these systems can extract information from natural lanugage or unstructured text. This increases their applicablility and the time taken to prepare data for advanced analyses.
For computer scientists out there – the worst case performance of a cognitive system for a database with n records and m fields could be O(nm^2), essentially that of the bruteforce method, but in reality if the model is trained well, it almost always performs near the best case, which assymptotes to O(n) with a very low constant. The longer the system is in operation, the lower the constant becomes.
Similarly, the data science driven approach to NLP is very different than traditional NLP stacks, where machine relies on inherent grammatical structure. Cognitive computers are faster to learn and can be very flexible with quality of input. Based on our experience, if there was a scale where completely structured data would be 1, and a well composed language piece 10, Coseer’s cognitive systems do the best between 6 and 9.
On the scale of 10, 10 being completely natural input and 1 completely structured, Coseer’s systems outshine between 6 and 9.
Overall, cognitive computing is very different from a big data or NLP based system. The artificial intelligence, and truly cognitive features in the design make it powerful, and especially suited for applications that run across structured and unstructured domains, and/or are dynamic in nature.