Navigating the Ups and Downs in the Field of AI
Over seven decades, the field of AI has experienced alternating cycles of heightened expectations and subsequent setbacks. However, a recent wave of advancements in generative AI has played a pivotal role in transforming these disappointments into promising growth opportunities. Amidst the current AI boom, where excitement and investment are at an all-time high, it is crucial to maintain a perspective that encompasses the challenges faced in the past. During those hard times, AI research struggled to gain the limelight it deserved. It faced significant hurdles in securing the necessary funding to advance its potential. Recognizing and reflecting upon these historical struggles helps us appreciate the progress made and underscores the importance of sustained support for future AI developments.
The first dawn
The first period of excitement, which began with the Dartmouth meeting around 1956, was later described by John McCarthy (the event’s main organizer) as the “Look, Ma, no hands!” era. During these early days, researchers built systems designed to refute claims of the forms “No machine could ever do X!” as such skeptical claims were common at the time. AI researchers created small systems that achieved X in a “microworld” to counter them. This provided a proof of concept and showed that X could, in principle, be done by machine. Tasks like mathematical question solving, theorems proofs, robotic arms, joke creation, etc.
First AI winter
The methods that produced successes in early demonstration systems often proved difficult to extend to a wider variety of problems or harder problems. One reason for this is that these methods explore the “combinatorial explosion” of possibilities that rely on something like an exhaustive search. Such methods work well for simple instances of a problem but fail when things get a bit more complicated. The performance of these early systems suffered because of poor methods of handling uncertainty, reliance on brittle and underground symbolic representations, data scarcity, and severe hardware limitations on memory capacity and processor speed. The realization that many AI projects could never make good on their initial promise led to the onset of the first “AI winter”: a period of retrenchment, during which funding decreased and skepticism increased and AI fell out of fashion.
The second dawn
A new springtime arrived in the early 1980s when Japan launched its Fifth-Generation Computer Systems Project which aimed to leapfrog the state of the art by developing a massively parallel computing architecture that would serve as a platform for artificial intelligence. The ensuing years saw a great proliferation of expert systems. Researchers designed expert systems as support tools for decision-makers. These rule-based programs made simple inferences from a knowledge base of facts. Human domain experts elicited these facts and meticulously hand-coded them in formal language. Builders constructed hundreds of these expert systems.
Second AI winter
However, the smaller systems provided little benefit, and the larger ones proved expensive to develop, validate and keep updated and were generally cumbersome to use. It was impractical to acquire a standalone computer just for the sake of running one program. By the late 1980s, this season, too, had run its course. A second “AI winter” descended. Private investors began to shun any venture carrying the “artificial intelligence” brand. Even among academics and their funders, “AI” became an unwanted epithet.
The Unstoppable advance in AI
Technical work continued apace, however, and by the 1990s, the second AI winter gradually thawed. Optimism was rekindled by the introduction of new techniques, which seemed to offer alternatives to the traditional logicist paradigm (often referred to as “Good Old-Fashioned AI”, or “GOFAI” for short), which had focused on high-level symbol manipulation and which had reached its apogee in the expert systems of the 1980s. The newly popular techniques, which included neural networks and genetic algorithms, promised to overcome some of the shortcomings of the GOFAI approach, in particular the “brittleness” that characterized classical AI programs.
The rise of Neural networks
The new techniques boasted a more organic performance. For example, neural networks exhibited the property of “graceful degradation”: a small amount of damage to a neural network. This typically resulted in a slight degradation of its performance, rather than a total crash. Even more importantly, neural networks could learn from experiences. Finding natural ways of generalizing from examples and finding hidden statistical patterns in their input. This made the nets good at pattern recognition and classification problems. For example, if we train a neural network on a dataset of sonar signals. We can teach it to distinguish the acoustic profiles of submarines, mines, and sea life. This can be done with greater accuracy than human experts. We can achieve this without requiring anyone to predefine the exact categories or weight different features beforehand.
After the introduction of the backpropagation algorithm, which enabled the training of multi-layered neural networks, the field experienced a renaissance. Although simple neural network models had been known since the late 1950s, their popularity soared due to this breakthrough. Such multilayered networks have one or more intermediary hidden layers of neurons between input and output layers. They can learn a much wider range of functions than their simple predecessors. Combined with the increasingly powerful computers that were becoming available, these algorithmic improvements enabled engineers to build neural networks that were good enough to be practically useful in many applications. The brain-like qualities of neural networks contrasted favorably with the rigidly logic-chopping but brittle performance or traditional rule-based GOFAI systems.
Genetic Algorithms: Unleashing Evolutionary Power
Evolution-based methods, such as genetic algorithms and genetic programming, constitute another approach whose emergence helped end the second AI winter. While it had a smaller academic impact compared to neural nets, evolutionary methods gained widespread popularity. In evolutionary models, we maintain a population of candidate solutions, which can consist of data structures or programs. We generate new candidate solutions randomly by mutating or recombining variants from the existing population. Such algorithms can produce efficient solutions to a very wide range of problems. Solutions that may be strikingly novel and unintuitive. Often looking more like natural structures than anything that a human engineer would design.
Conclusive Thoughts on AI Developments
Neural networks and genetic algorithms revolutionized the field of AI in the 1990s. They captured widespread attention as they presented promising alternatives to the stagnating GOFAI (Good Old-Fashioned AI) paradigm. Fast forward to today, and these algorithms still form the fundamental bedrock of numerous advancements in AI ML solutions. Gaining a comprehensive understanding of these methods can greatly enhance our comprehension of the underlying technology driving modern AI solutions. By doing so, we avoid the fallacy of perceiving these solutions as magical black boxes. We can instead recognize them as sophisticated products of years of persistent challenges and continuous improvements.
Help links
- Example reference to training a neural network. To distinguish the acoustic profiles of submarines, mines, and sea life as mentioned in the above text. Github link
Acknowledgments
- The text referred from Chapter 1 of Nick Bostrom’s book Superintelligence – Paths, Dangers, Strategies.
Blog Sponsor
This blog post is sponsored by Convosense.ai
Unlock the full potential of your customer conversations with Convosense, the ultimate AI-powered conversation analytics platform. Maximize sales, improve customer satisfaction, and prevent churn like never before. Convosense integrates seamlessly with any CRM, empowering your team with powerful analytics and actionable insights to take your business to new heights.
For more info visit – Convosense.