30 January 2015
Events and the dependencies between them may be represented using a form of artificial intelligence known as Artificial Neural Networks (ANN). ANNs are inspired by a simplified version of the brain consisting of neurons (represented by events) and synapses (represented by dependencies or connections between events). ANNs provide a flexible model that can be scaled up massively to represent complex simulations of the real world – it lends very well to parallel computing, uses memory and storage spaces in a structured and manageable manner, and may be compartmentally implemented in a modular manner within a distributed or server farm environment. Coupled with other aspects of the architectural design, the neural network may be combined with fuzzy logic, hierarchically multidimensional representations of data, persistent storage and etc to deliver the required features and functions of the game engine.
ANN offer a distinct advantage of other common ways of implementing simulation games such as the approach using arrays in Games of Life which are widely used in simple simulation games such as Sim City, or the relatively unstructured approach of defining rules, players and random seeds such as the Monte Carlo Method which can grow in an unwieldy manner when a simulation is scaled up to reflect more complex models. It expands on the basic ideas of Bayesian Belief Networks and provides a foundation to introduce value-added peripheral features to handle more complex models Wikipedia quotes:
Like other machine learning methods – systems that learn from data – neural networks have been used to solve a wide variety of tasks that are hard to solve using ordinary rule-based programming, including computer vision and speech recognition.
Wikipedia introduces ANNs as
In machine learning, artificial neural networks (ANNs) are a family of statistical learning algorithms inspired by biological neural networks (thecentral nervous systems of animals, in particular the brain) and are used to estimate or approximate functions that can depend on a large number of inputs and are generally unknown. Artificial neural networks are generally presented as systems of interconnected “neurons” which can compute values from inputs, and are capable of machine learning as well as pattern recognition thanks to their adaptive nature. For example, a neural network for handwriting recognition is defined by a set of input neurons which may be activated by the pixels of an input image. After being weighted and transformed by a function (determined by the network’s designer), the activations of these neurons are then passed on to other neurons. This process is repeated until finally, an output neuron is activated. This determines which character was read.
Note that Wikipedia described ANNs as a family of algorithms, not a single algorithm. This reflects the fact that there are many types and variations of ANNs offering a rich variety of solutions to produce simulation games, for example ANNs that implement Back Propagations may be used to produce games that try to fit a simulation model from a gross estimation of a real world scenario into a closer fit of known historical reality, thereby discovering insights into relationships between influencing events that reflect the real world. In a real sense, the use of ANNs in simulation games allow not only learning by human students but also unsupervised or semi-supervised learning by machines, the outcome of which will reinforces human learning through the process of discovering hitherto unexpected insights or confirming intuitive expectations.
In particular the manner in which ANNs may be used to build up an arsenal of discoveries from a relatively small set of known assumptions in a non-linear fashion based on statistics lend itself very well in modelling complex, real world situations.
Wikipedia further describes ANNs as follows
A Simple 3-Layered Artificial Neural Network
Examinations of the human’s central nervous system inspired the concept of neural networks. In an Artificial Neural Network, simple artificial nodes, known as “neurons“, “neurodes”, “processing elements” or “units”, are connected together to form a network which mimics a biological neural network.
There is no single formal definition of what an artificial neural network is. However, a class of statistical models may commonly be called “Neural” if they possess the following characteristics:
- consist of sets ofadaptive weights, i.e. numerical parameters that are tuned by a learning algorithm, and
- are capable ofapproximating non-linear functions of their inputs.
The adaptive weights are conceptually connection strengths between neurons, which are activated during training and prediction.
Neural networks are similar to biological neural networks in performing functions collectively and in parallel by the units, rather than there being a clear delineation of subtasks to which various units are assigned. The term “neural network” usually refers to models employed in statistics, cognitive psychology and artificial intelligence. Neural network models which emulate the central nervous system are part oftheoretical neuroscience and computational neuroscience.
In modern software implementations of artificial neural networks, the approach inspired by biology has been largely abandoned for a more practical approach based on statistics and signal processing. In some of these systems, neural networks or parts of neural networks (like artificial neurons) form components in larger systems that combine both adaptive and non-adaptive elements. While the more general approach of such systems is more suitable for real-world problem solving, it has little to do with the traditional artificial intelligence connectionist models. What they do have in common, however, is the principle of non-linear, distributed, parallel and local processing and adaptation. Historically, the use of neural networks models marked a paradigm shift in the late eighties from high-level (symbolic) artificial intelligence, characterized by expert systems with knowledge embodied in if-then rules, to low-level (sub-symbolic) machine learning, characterized by knowledge embodied in the parameters of a dynamical system.
In particular, advances in ANNs has led to the development of a particular field in artificial intelligence or knowledge discovery known as Deep Learning.
Wikipedia describes Deep Learning as follows:
Deep learning (deep structured learning or hierarchical learning) is a set of algorithms in machine learning that attempt to model high-level abstractions in data by using model architectures composed of multiple non-linear transformations.
Deep learning is part of a broader family of machine learning methods based on learning representations of data. An observation (e.g., an image) can be represented in many ways such as a vector of intensity values per pixel, or in a more abstract way as a set of edges, regions of particular shape, etc. Some representations make it easier to learn tasks (e.g. face recognition) from examples. One of the promises of deep learning is replacing handcrafted features with efficient algorithms for unsupervised or semi-supervised feature learning and hierarchical feature extraction.
Research in this area attempts to make better representations and create models to learn these representations from large-scale unlabeled data. Some of the representations are inspired by advances in neuroscience and are loosely based on interpretation of information processing and communication patterns in a nervous system, such as neural coding which attempts to define a relationship between the stimulus and the neuronal responses and the relationship among the electrical activity of the neurons in the brain.
Various deep learning architectures such as deep neural networks, convolutional deep neural networks, and deep belief networks have been applied to fields like computer vision, automatic speech recognition, natural language processing, audio recognition and bioinformatics where they have been shown to produce state-of-the-art results on various tasks.
Alternatively, deep learning has been characterized as a buzzword, or a rebranding of neural networks
ANNs has been widely used in the Medical and Robotics fields in particular due to its pattern recognition and discovery capabilities. The application of ANNs to simulate and predict real world business is also widely cited by numerous parties, such as the following:
Applications of Neural Networks
Neural Networks in Business and Economy
Many problems in business world are essentially about trying to predict the likelihood of different outcomes. The real world is so complex, with so many inter-related variables that predicting these outcomes are often very difficult. Conventional computing has a limited role to play in helping with these tasks but neural networks offer an alternate solution. By feeding in the different factors that affect an outcome over time a network can analyse previous trends and patterns to predict the future.
Predicting Stocks with Neural Networks
Warren Buffett is one of the most important and successful figures in the financial world. Others have built their own highly successful investment portfolios using his theories on investment and market analysis. Walkrich Investment Advisors used Neural Networks to produce an investment tool WRRAT based loosely on Warren Buffett’s ideas to predict stock prices, and determine which stocks are trading below their market value. The results from January 1995 to January 1996 showed that a Portfolio of WRRAT’s most under-priced shares saw an average advance of 33%.
Another example is the use of neural network software by LBS Capital Management to predict the S&P 500 index. The company uses an expert system to provide instructions to the neural network, which then processes the data accordingly. When tested with hundreds of previous days data the neural network LBS trained predicts the S&P 500 with an accuracy of about 95%.
Predicting Currencies with Neural Networks
O’Sullivan Investments successfully used many neural networks in order to advise them of market trends. Mr James O’Sullivan produced an article Neural Nets: A Practical Primer, AI In Finance, 1994 outlined some of the networks used. One of the most important factors in producing a successful net is to ask the right kind of question. Rather than simply ask what the projected price of a currency might be, he asks at what price the market is likely to take off in one direction or the other etc.
Predicting Natural Gas Prices with Neural Networks
Northern Natural Gas is a regulated wholesaler of natural gas. They must develop and file a rate for the gas they sell based on the average cost of the gas. By developed a neural network that use factors such as the quarter of the year, season, temperature last month etc. to predict the following months oil price, the company was better able to plan rates.
Predicting bonds with Neural Networks
- R. Pugh & Company does consulting to predict the prices of bonds of public utilities. The company used neural networks to help forecast the following years corporate bond prices and ratings of over 100 public utility companies. The network they used compared very favourably to conventional mathematical analysis. Whereas the network was able to predict a utilities rating (A, B, C) with 95% accuracy, conventional mathematical analysis was only effective 85% of the time. The only difficulties encountered by the network were associated with companies experiencing particularly unusual problems that were not incorporated into the networks inputs.
Targeting Direct Mail Marketing with Neural Networks
Microsoft used neural networks to maximise the effectiveness of their marketing campaign. Each year the company sent out mail to its registered customers. Most of this mail offered upgrades or new software but the response rate was rather low. The company used a neural network that was fed various variables such as how recently they registered, how many products they have bought etc. to target users more effectively. The results showed an average mailing lead to a 35% cost savings.
Credit Scoring with Neural Networks
Research conducted by Dr Herbert Jensen PhD demonstrated that “building a neural network capable of analysing the credit worthiness of loan applicants is quite practical and can be done quite easily”. The neural network was trained on no more than 100 loan applications to process application data such as occupation, years with employer etc. Despite the relatively small training set the network achieved a 75-80% success rate. This compared well with more traditional scoring methods that resulted in about a 75% success rate.
Real Estate Appraisal with Neural Networks
Several neural networks have been used to predict the sale prices of homes in order to help appraisers assess, sellers estimate asking prices, and home owners decide on improvements. Richard Borst successfully trained a neural network to appraise real estate in the New York area. His network incorporated almost 20 variables including the square feet of living area, age, etc. He used over 200 sales records from 1988 and 1989 to train the network with about 90% accuracy.
Neural Networks in business
Business is a diverted field with several general areas of specialisation such as accounting or financial analysis. Almost any neural network application would fit into one business area or financial analysis.
There is some potential for using neural networks for business purposes, including resource allocation and scheduling. There is also a strong potential for using neural networks for database mining, that is, searching for patterns implicit within the explicitly stored information in databases. Most of the funded work in this area is classified as proprietary. Thus, it is not possible to report on the full extent of the work going on. Most work is applying neural networks, such as the Hopfield-Tank network for optimization and scheduling.
There is a marketing application which has been integrated with a neural network system. The Airline Marketing Tactician (a trademark abbreviated as AMT) is a computer system made of various intelligent technologies including expert systems. A feedforward neural network is integrated with the AMT and was trained using back-propagation to assist the marketing control of airline seat allocations. The adaptive neural approach was amenable to rule expression. Additionaly, the application’s environment changed rapidly and constantly, which required a continuously adaptive solution. The system is used to monitor and recommend booking advice for each departure. Such information has a direct impact on the profitability of an airline and can provide a technological advantage for users of the system. [Hutchison & Stephens, 1987]
While it is significant that neural networks have been applied to this problem, it is also important to see that this intelligent technology can be integrated with expert systems and other approaches to make a functional system. Neural networks were used to discover the influence of undefined interactions by the various variables. While these interactions were not defined, they were used by the neural system to develop useful conclusions. It is also noteworthy to see that neural networks can influence the bottom line.
The HNC company, founded by Robert Hecht-Nielsen, has developed several neural network applications. One of them is the Credit Scoring system which increase the profitability of the existing model up to 27%. The HNC neural systems were also applied to mortgage screening. A neural network automated mortgage insurance underwritting system was developed by the Nestor Company. This system was trained with 5048 applications of which 2597 were certified. The data related to property and borrower qualifications. In a conservative mode the system agreed on the underwritters on 97% of the cases. In the liberal model the system agreed 84% of the cases. This is system run on an Apollo DN3000 and used 250K memory while processing a case file in approximately 1 sec.
A very comprehensive paper on applications of ANNs including in economics and business citing amongst others Quality Control, Market Analysis and Financial Forecasting and Portfolio Management (amongst numerous others) may be accesses from http://www-isl.stanford.edu/~widrow/papers/j1994neuralnetworks.pdf
The following paper provide some in depth examples of the application of ANNs in business. Although the examples do not apply ANNs to business simulations per se, it is quite easy to see how the same mechanisms may be applied to perform simulation games – https://www.google.com.my/url?sa=t&rct=j&q=&esrc=s&source=web&cd=9&ved=0CFUQFjAI&url=http%3A%2F%2Fwww.cse.hcmut.edu.vn%2F~dtanh%2Fdownload%2FANN_BusinessApplications.ppt&ei=8z3LVOH9FcHUmAX89YKoBw&usg=AFQjCNG8XKuZ5v6kF33rery0Jvm0I1fM8g&sig2=oHSWHTCevf9KOi–IvdUsQ&bvm=bv.84607526,d.dGY&cad=rjt