An Evolutionary Complex Systems Decision-Support Tool for the Management of Operations

This research aimed to add both to the development of complex systems thinking in the subject area of Operations and Production Management and to the limited number of applications of computational models and simulations from the science of complex systems. The latter potentially offer helpful decision-support tools for operations and production managers. A mechanical engineering firm was used as a case study where a combined qualitative and quantitative methodological approach was employed to extract the required data from four senior managers. Company performance measures as well as firm technologies, practices and policies, and their relation and interaction with one another, were elicited. The data were subjected to an evolutionary complex systems model resulting in a series of simulations. The findings included both reassuring and some unexpected results. The simulation based on the CEO's opinions led the most cohesive and synergistic collection of practices describing the firm, closely followed by the Marketing and R&D Managers. The Manufacturing Manager's responses led to the most extreme evolutionary trajectory where the integrity of the entire firm came into question particularly when considering how employees were utilised. By drawing directly from the opinions and views of managers rather than from logical 'if-then' rules and averaged mathematical representations of agents that characterise agent-based and other self-organisational models, this work builds on previous applications by capturing a micro-level description of diversity and a learning effect that has been problematical not only in terms of theory but also in application. This approach can be used as a decision-support tool for operations and other managers providing a forum with which to explore a) the strengths, weaknesses and consequences of different decision-making capacities within the firm; b) the introduction of new manufacturing technologies, practices and policies; and, c) the different evolutionary trajectories that a firm can take.


Introduction
The aim of this research was twofold: to model the diversity in management decision-making and the effects on the potential evolutionary trajectories that a firm can take, and in turn, to demonstrate the usefulness of a new modelling approach which could form the basis for a decision-support tool for the management of operations. The paper begins by highlighting, in the context of recent advances in complex systems thinking, modelling and simulation, the role and importance of management decision-making in organisational evolution along with the significance of diversity among individuals 1 To whom correspondence should be addressed. that make the decisions. A particular type of modelling and simulation technique, based on evolutionary complex systems theory, is then proposed as being the most appropriate as it is the only approach that fully incorporates learning along with diversity at all levels of description. Following an account and justification of the research methods that were employed, simulations of the outcomes of management decision-making are then presented along with a discussion of the significance both in their own right and in the context of the literature. The paper concludes with some closing remarks on the research and practical implications of this work. Supply networks, organisations, people and decision-making processes are increasingly being viewed and treated as complex systems [1][2][3][4][5]. There has also been a growing trend recently, and particularly during the last decade or so, in the modelling and simulation of such systems [6][7][8][9][10][11]. With complex systems, evolution and change are inevitable and the performance and survival of a firm is largely determined by its management of change [5,12,13]; and with manufacturing firms, technological change in particular [14]. This has prompted a call for increased understanding of the underlying processes in the adoption and implementation of new technologies [15][16][17], practices [18][19][20][21][22] and policies [13,23,24]. A convincing argument can be made that management decision-making takes a central role.
Raymond et al [14], for example, whilst investigating technology adoption in SMEs, identified the decision-making process as one of three main profiles of strategic advantage along with technological expertise and organisational capabilities. Karkkainen and Hallikas [25] expanded this outlook and explored, via case-study, the dynamics of inter-organisational network-related decision-making under a series of scenarios relating to risk management, learning and the business environment. They pointed out that not only was there a dearth of research in management decision-making processes underlying organisational change but that a holistic and systemic approach was needed to fully understand both intra-and inter-organisational decision-making. In remedying this, Meade et al [26] applied the theories of chaos and complexity to provide an understanding of the decisions behind formulating strategies for the successful positioning of products in the technology adoption life-cycle. Using several case studies of firms within the ICT industry, this work successfully demonstrated the usefulness of this approach. Similarly, Nilsson and Darley [10], after justifying the use of the complex adaptive systems perspective, applied agent-based modelling [see also 7,8,27] to not only further understanding in the decision-making process but also to provide decision-makers with 'what-if' scenarios where the consequences of particular decisions can be analysed and evaluated. This study is important in that it was the first to produce empirically verified results of agent-based models in the subject area of Operations and Production Management. Whilst acknowledging that management decision-making is central to any organisational change/transformation, there is also rising awareness of the impact of diversity in the decision-making process [28,29].
To elaborate, there has long been a recognition of the differences in motivations, interests and priorities of managers from different functional areas [23,30] which if not managed appropriately can be a significant detriment to firm performance [31]. Getting a balance is key, as diversity can also be hugely advantageous in terms of innovative capabilities if set within a conducive organisational culture [12]. Diversity, in the context of this research, refers not only to the different decisions that can and are made but to the range of approaches taken when decision-making due to individuals' different perspectives, beliefs, attitudes and information-processing capacities/abilities [28]. Simon's [32,33] notion of bounded rationality from the cognitive sciences and more recently evolutionary economics, which refers to the incomplete knowledge that people have and use (and the misuse of information) when decision-making, is also relevant here. This important element adds to heterogeneity and these diverse views have the potential to radically affect the evolution of a company particularly in times of change, for example, when introducing a new technology, practice or policy [12,17]. Capturing this level of diversity has been problematical not only in terms of theory [but see, for example, 34,35,36] but also in application and perhaps more evidently.
Evolutionary complex systems modelling departs from the other computational modelling techniques, such as system dynamics, agent-based models and other self-organisational models, as it incorporates the role and influence of micro-diversity along with experiential learning which are arguably the driving forces and impetus behind change that is evolutionary in nature [28]. Evolutionary complex systems theory is a European branch of complexity thinking stemming from Prigogine's [37] Nobel Prize winning work. The approach has now been successfully applied to ecosystems, urban systems, economic markets and, more recently, in evolving an entire industry [17,28,38,39].
To illustrate the differences between the different systems models a consideration of underlying assumptions is needed. Modelling assumptions create a hierarchy of models from known certainties and perfect prediction through to explorations of the unknown and the least-likely of potentialities [39]. All systems models have at least two assumptions: 1) that a boundary exists between the system, in this case the firm, and its environment, and 2) that the system's components, e.g., the firm's technologies, practices and policies, can be classified to produce a taxonomy.
Additional assumptions concern the system's components and their interactions. System dynamic models have components and interactions that represent the average. When everything has been averaged there is just one future path -the most probable. These models give the impression of complete understanding and knowledge, and in turn perfect predictability. But what do average components and interactions actually mean when applied to a firm? Take, for example, the implementation of line balancing or employee empowerment. Not every firm has the same approach. Indeed, if every firm's approach was scrutinised, there would be a high degree of idiosyncrasy. Furthermore, and in terms of interaction with other components (e.g. quality or inspection policies), not all implementations of line balancing and/or employee empowerment have the same outcomes.
That is, what would work in one organisation, or even one point in time, may not necessarily work in another.
By introducing the non-average, the nature of the model begins to change from certainty and prediction to exploration and potentialities. Through the inclusion of all the potential types of interactions that can occur, models can begin to explore and reach many possible future scenarios through self-organisational processes (e.g. the different outcomes of line balancing or employee empowerment implementation in the example above). The few complex system computational models found in the literature are of this type [see, for example, 6, 7-11, 26, 27]. Although highly useful both practically and intellectually, there is, nonetheless, a limitation to these models particularly in their application to social systems. To represent diversity, these models have a stochastic mechanism that generates 'noise', which is certainly more realistic than assuming only average conditions and interactions.
However, whilst these models introduce non-average interactions, the components/agents are still of an average type that are subject to a pre-defined, 'if-then' rule-based system. The noisy interaction of 'average' elements does not have the same outcome, or spread of outcomes as the interaction of diverse, heterogeneous individual elements. In the former case we may see different configurations or regimes of operation appearing, but in the latter case, new combinations of different elements leading to new, emergent capabilities and dimensions of performance can occur. This is the essential characteristic of evolutionary change -a process of qualitative, not just quantitative, change.
It is heterogeneous components that distinguish evolutionary complex systems models from selforganisational models [39]. Representing all possible types of components, through the introduction of internal or micro-diversity, as well as all possible interactions produces a more realistic representation of true evolutionary processes. Whilst blind adaptation is associated with self-organisational models, evolutionary complex systems models mimic co-evolution through experiential learning. Control devolves fully from the global/system level to the local/individual level and is an expression of singular behaviours and their performance/success relative to others within the system; evolution proceeds through de-centralised, rather than centralised, decision-making. In addition, evolution, being an open process, ensues through a combination of both the determinism of the individuals' purposefulness but also by chance events. The intake of new kinds of individuals, or the changing views and thoughts that they may have is not a rational, calculated process because the implications of any particular heterogeneity is not known until after an evolutionary step has occurred in system behaviour. In this way, the evolution that does occur is not really predictable but results from the interplay of individual heterogeneity and the differential performances of the resulting organisations. This is largely characterised by an inevitable lack of pre-existing knowledge of the link between individual and system behaviour and can be thought of as resulting from a degree of 'error-making' [28]. The role of chance is fundamental, however, creating a rich medium for experiential learning through ceaseless experiments in behaviour space [39]. As the approach incorporates diversity at all levels of description, it is arguably the most appropriate when trying to attain a better understanding of the role of diversity in management decision-making and the impacts on a firm's evolutionary trajectories.

Research Methods
The research aim was achieved by: a) building a profile of a company through a case-study approach involving observation and semi-structured interviews; b) gauging, via a quantitative questionnaire, the respondents' perception of how the firm's technologies, practices and policies interacted with one another; and then c) subjecting the data to the evolutionary complex systems computational model. In doing so, it was possible to compare and contrast the views of different people in one organisation and examine the consequences of decision-making from people that inevitably have different opinions and base their decisions on different information, values and beliefs. The case-study approach was selected to better illustrate and exemplify the utility of the evolutionary complex systems modelling technique [40,41] and is consistent with similar research [10,26]. As is common with case study research, sampling was purposive [42]. That is, a number of firms were pre-screened to determine whether they had a suitable profile that would help achieve the research aim. A mechanical engineering firm was approached and consented to act as the case study. The CEO, and three directors, responsible for Sales, Manufacturing and Quality, and Research and Development, participated. Interviews were based on a semi-structured qualitative questionnaire. With the internal validity of the questionnaire in mind [42], the interview schedule was first piloted on both an two industrialists and an academic familiar with Operations and Production Management after which small adjustments were made. Interviewer and interviewee biases were minimised through the selection of neutral settings for interviews and through a standardised presentation of non-leading questions [42]. The interview schedule was designed to: a) enable a discussion of what the participants deemed important for the survival of the company in terms of performance; and b) encourage an exploration of what characterised the company in terms of the firm's technologies, practices and policies (henceforth referred to as 'character-states') covering, for example, the workforce, suppliers, quality, R&D and factory layout. From the interviews, 4 main performance criteria, i.e., product quality, cost efficiency, relationships with clients, and schedule adherence, and 25 company characterstates (listed in table 1) were identified and provided the basis for a quantitative questionnaire. The performance criteria are largely consistent with both the literature and previous empirical research on manufacturing performance [15, 18,   Employee empowerment ERP system to organise and monitor resources Visibility of resources through stock to product Automated set-ups in machines Automated cells with robots Set-up time reduction Resources flexibility and priority control 24 hours of non-stop production Traceability along the production process MRP to replenish material ERP integrated with clients' ERP Co-operation with suppliers Investment on R&D and equipment The questionnaire was designed to gather the directors' views of how the 25 character-states interacted with one-another in relation to the overall performance of the firm. Piloting, involving two academics and two industrialists, was in two stages after which incremental adjustments were made to the questionnaire to ensure construct validity [42]. The final questionnaire had three parts. The first asked participants to rank the overall importance of the 4 performance criteria to the company, for example, 1 st : external relationships; 2 nd : schedule adherence, and so on. The second part asked participants to rank the impact or strength of association of each of the character-states on each of the performance criteria; for example, CS 1, international quality standards, may be associated 1 st with product quality, 2 nd with relationships with clients, 3 rd with cost efficiency and 4 th with schedule adherence, etc. These first two sections aimed to provide a basis for weighting mechanisms in the model as some character-states impact particular performance criteria more than others. The final part of the questionnaire asked participants to gauge the interactions between each of the character-states in terms of the overall performance of the firm. The answer options were based on a 7-point Likert scale (-3 to +3) determining the degree of positive/neutral/negative interactivity. That is, participants could indicate a moderately synergistic interaction between, for example, CS 1 and CS 2 as '+2' and a strongly antagonistic interaction between CS 1 and CS 3 as '-3' and so on. The evolutionary complex systems model drew directly from this latter questionnaire part.

The Simulation Model
The model was designed to simulate the interaction between the firm's character-states drawing directly from the four decision-makers' perceptions, values, and beliefs, etc. The model is based on a mathematical expression that describes how a change can occur and a new behaviour be adopted in the system [39]. Behaviours may represent for example new species in a natural ecosystem or the introduction of new technologies, practices and policies in a manufacturing system. A new practice or behaviour can only grow in the system if the system is unstable to its appearance and this therefore gives a mathematical expression corresponding to its growth or decline. This allows the model to explore how the selection mechanisms will respond to any particular succession of new practices or 2 Note: 'character-states' will be abbreviated in the text to 'CS' when referred together with a particular technology, practice or policy; for example 'CS 1, International Quality Standards' behaviours, as this evolutionary process will only allow the retention of certain, effective bundles of practices and behaviours. These emergent bundles of practices and behaviours will experience positive feedback and growth when their particular characteristic performances correspond to that which the selection environment requires. These might be, for example, increased sales, quality, productivity and adherence to schedules, and the selection environment will ensure that the appropriate assemblages of practices and behaviours will emerge. It is through this parallel pursuit of multiple possible pathways that the potential customers' requirements and, at the same time, organisations will be created that achieve what is required. If firms do not experiment with new practices and behaviours, then they will be "outrun" by at least some of those that do, and will be eliminated. However, it is the ignorance and uncertainty about the effectiveness and outcomes to be expected from different combinations of practices that mean that different sequences will be tried and different bundles emerge. On the other hand, if firms try out too many new practices and behaviours at the same time, then there will be an internal competition between different possible organisational structures and emergent performances and so this too will be eliminated in a competitive system. In addition, the fact that there will be any emergent successful capabilities is not known until after it has occurred and so if repeated launching of new practices and behaviours does not create new, successful production then it is time to move investment to other things. This model captures the idea that experimental explorations will either discover and create market niches or that there do not seem to be any and the investment is wasted. Evolution of the system is either retained in exploring and finding potential demand hidden in a particular domain, or it will move on to another domain.
The model is based on the equations given in Allen et al [39], developed in Turbo Basic® and run in the Microsoft Dos® operating system. Several variables can be manipulated and calibrated, three of which require more explanation. The first is the running time of the simulation. This may be adjusted to permit finding stable solutions which are typically found within 10-50,000 arbitrary time units. The second variable is the number of character-state initiations in the model which allows exploration of particular organisational forms. The third is the performance value of the character-state. Values may lie between 0-30 arbitrary units. The higher the value, the better the performance and importance within the organisation.
Each bar in the simulations (see figures 1-5) represents a character-state with the number indicated below. The presentation of results has been simplified using Microsoft Excel®. This is to enhance interpretation with the reader in mind. The height of each bar is an indication of the value to the firm of that particular character-state relative to the other character-states in terms of the performance criteria. The simulations presented here launched the character-states with a starting value of 5 units. Before the results are presented there are two qualifications that need to be highlighted. The first is that the particular simulations presented here are among many possible trajectories dependent on initial conditions. All simulations of the different directors, however, begin with identical initial conditions. In addition, the simulations selected were deemed fairly representative of a series of repeat simulations. The second qualification is that, unlike reality, the simulations depict an evolution of the firm with all character-states starting as equal (with a value of 5 units). Nonetheless, the results from this procedure do highlight differences (and similarities) between the different decision-makers.

Results and Discussion
To fully appreciate the effects of diverse decision-making capacities, five simulations are presented. To summarise the first, figure 1 depicts, in simplified form, an evolutionary trajectory of the firm when all the decision-makers' opinions are aggregated together. Twenty out of the twenty-five character-states increased their standing in terms of value at the final stable solution. Nine characterstates equalled or exceeded 8 value units and two, CS 1, international quality standards, and CS 5, operator involvement in error detection/correction, reached double figures. CS 21, traceability along the production process, kept its initial value, whilst CS 4, preventive maintenance, CS 10, line balancing, and CS 15, visibility of resources through stock to product, declined by 1, 2 and 3 value  There are several significant points to discuss here. The first concerns the nature of the model, the micro-diversity that has been captured and the potential insights that this gives. As can be seen from the research methods, the model draws directly from the opinions and views of managers rather than from logical 'if-then' rules and averaged mathematical representations of agents that characterise agent-based and other self-organisational models, particularly those proposed by Lim and Zhang [8], Zhou et al [7], Li et al [9], and Poundarikapuram and Veeramani [29]. The second point is that the simulations reveal management concerns, both collectively (in terms of team decision-making) and individually, that can then be flagged up for further discussion and exploration, similar to the 'what-if' scenario building of Nilsson and Darley's [10] work. Taking the collective concerns as an illustration, when the managers' opinions are aggregated, CS 6, corrective maintenance subcontracted, fails, pointing to some common belief or opinion that there is a problem with this practice, for example, that it has had unsatisfactory outcomes in the past or has incurred significant costs. The simulation also flags up concerns over CS 4, preventive maintenance (which is somewhat contradictory given the failure of CS 6, corrective maintenance subcontracted) CS 10, line balancing and CS 15, visibility of resources through stock to product as these performed poorly with respect to the other characterstates. With these findings in mind, the overall integrity of the organisation was good, however, suggesting that as a group the decision-makers work fairly cohesively. The next step in the research procedure was to analyse the opinions of the different decisionmakers separately and compare and contrast the results. Figure 2 shows a simulation of the CEO's opinions at the stable solution. During this simulation, nineteen of the twenty-five character-states increased on their initial value with ten character-states equalling or exceeding 8 value units. CS 1, international quality standard, and CS 5, operator involvement in error detection/correction, again reached double figures. CS 18, set-up time reduction, and CS 21, traceability along the production process, remained at their original value. As with the first simulation, CS 4, preventive maintenance, CS 6, corrective maintenance subcontracted, and CS 15, visibility of resources through stock to product, declined in value but this time were joined by CS 11, operator rotation, which stabilised at 4 units. No character-state failed during this simulation. Interestingly, the CEO's simulation mirrors the first simulation where all the directors' opinions are aggregated. How can this be interpreted? Does the CEO have a balanced view of the workings of the organisation and whose opinions reflect the 'middle ground' of the decision-makers? Or does the CEO have the ability to project his understanding/influence onto the other directors in their particular fields of expertise? This may reflect in some respects, for example, what Jarratt [12] sees as achieving the right balance between centralised and de-centralised systems in the management of diversity. What is also interesting is that the CEO's simulation was the only one without any character-state failures. This gives a strong impression that the CEO does have a balanced view of the organisation and in contrast to the other directors sees how all the company policies, practices and technologies fit and work together. However, the CEO did appear to have concerns over CS 4, preventive maintenance, CS 6, corrective maintenance subcontracted, CS 11 operator rotation and CS 15 visibility of resources through stock to product, which, apart from CS 11 operator rotation, reflects the simulation of the aggregated opinions. Figure 3 portrays a stable solution for the simulation of the Sales Director's opinion scores. To summarise, during this simulation, seventeen of the twenty-five character-states increased on their initial value, with nine equalling or exceeding 8 value units. Three, CS 1, international quality standards, CS 5, operator involvement in error detection/correction, and CS 24, co-operation with suppliers, reached double figures. In this instance, five character-states, i.e. CS 4, preventive maintenance, CS 13, employee empowerment, CS 16, automated set-ups in machines, CS 17, automated cells with robots, and CS 22, MRP to replenish material, remained at their original values whilst CS 15, visibility of resources through stock to product, fell by 1 value unit. Two characterstates, i.e. CS 6, corrective maintenance, and CS 10, line balancing, failed altogether. This simulation begins to demonstrate the significant role that individual differences or diversity has in the management decision-making process and add support to the arguments of Allen et al [28], Poundarikapuram and Veeramani [29], and O'Leary-Kelly and Flores [48]. In the Sales Director's simulation there are several differences that need to be highlighted. The first is that only seventeen of the twenty-five character-states improved on their starting value. This is in contrast to the first two simulations where there were improvements for twenty of the aggregated scores and nineteen of the CEO's scores. However, seven of the nine character-states that surpassed 8 value units agreed with the CEO. The main difference, particularly in terms of the CEO's results, was that two character-states, CS 6, corrective maintenance subcontracted, and CS 10, line balancing, failed altogether. Although the performance of the former character-state reflects the aggregated results and to a degree the CEO's simulation, the latter is opposed to the opinion of the CEO. Why the Sales Director thinks negatively of line balancing may be a symptom of functional barriers [30,31] and is another area for further management analysis. The Research and Development Director's simulation results at a final stable solution are shown in figure 4. To sum up, nineteen out of twenty-five character-states increased on their starting value. This time, ten character-states reached values of 8 units or more with two, CS 1, international quality standards, and CS 24, co-operation with suppliers exceeding double figures. Only one, CS 9, multiskilled workers, remained constant. CS 4, preventive maintenance, and CS 22, MRP to replenish material, declined in value with both losing approximately 2 value units. Furthermore, CS 6, corrective maintenance subcontracted, CS 14, ERP system, and CS 15, visibility of resources through stock to product, all failed to make an impact during the simulation and disappeared.  The Manufacturing and Quality Director's simulation proved to be the most interesting as it was in stark contrast to the rest of the decision-makers' simulations and is the best example of the significant role and impact of diversity among decision-makers alluded to by Nilsson and Darley [10], Allen et al, [28] and Jarratt [12]. The most noticeable difference was that six of the twenty-five character-states disappeared from the organisation with a further two finishing with nominal values (under 2 units). Of the eight character-states that failed or under performed, six related to workforce policies (i.e. multiskilled workforce, line balancing, operator rotation, operator flexibility, empowerment, and 24 hour working pattern). This pattern suggests that the Manufacturing and Quality Director has issues with the way the workforce is utilised. Suggested reasons could be that the workforce policies negatively affect product quality or that the director prefers different policies. With only fifteen character-states gaining on the original values, it was, however, interesting that twelve of these reached or exceeded 8 value units, which was the most out of all simulations. With both the failures and high scoring character-states, this simulation represents the most extreme potential evolutionary trajectory of the firm out of the five presented here. This simulation when compared to the other simulations also lends significant support for a long standing call voiced by O'Leary-Kelly and Flores' [23] and Malhotra and Sharma [31] for more integration and understanding between functional areas and particularly between the manufacturing/quality and other functional areas.
On a more general note, there was a fairly good agreement between all of the decision-makers regarding the importance of several character-states with the following exceeding 8 value units in most of the simulations: CS 1, International quality standards, CS 2, standards for purchasing resources, CS 3, 100% inspection, CS 5, operator involvement in error detection/correction, CS 7, 5Ss programme, CS 16, automated set-ups in machines, CS 23, ERP integrated with clients' ERP, CS 24, co-operation with suppliers and CS25, investment on R&D and equipment. The practical usefulness perhaps lies more in the exploration of the more problematical areas [10]. The character-states with the most problems (determined by their failure in at least two simulations) included CS 6, corrective maintenance subcontracted, which failed in three simulations and CS 10, line balancing which failed in 2 simulations. The simulations in general, together with the Manufacturing and Quality Director's opinions, also suggest that the workforce policies need further scrutiny as these character-states had mediocre performances and rarely exceeded the 8 value unit mark. In terms of methodology, the findings also strengthen the consistency/reliability of the data collection procedure adapted from previous work [17,28,38,39].

Closing Remarks
This project aimed to provide insights into the potential evolutionary effects of the diversity in management decision-making and attempted to add not only to the theoretical development of complex systems thinking but perhaps more so to the application of computational models and simulations which is still arguably lacking in the subject area of Operations and Production Management [5,10,13]. To build on this, further work and clarification is required into the potential underlying reasons and consequences for the successes and failures of particular technologies, practices and policies. More case studies are also required firstly, to strengthen the reliability and validity of the methods employed and secondly, to encompass more management decision-making scenarios.
In terms of practical value, evolutionary complex systems models and simulations, along with other similar tools such as those advocated by Nilsson and Darley [10] and Meade et al [26], offer a more realistic decision-support tool for management with which to explore the strengths, weaknesses and consequences of different decision-making capacities within the firm. In this research, for example, out of the four decision-makers, the CEO appeared to have the most balanced view of the organisation as all character-states successfully survived. Both the Sales and Research and Development Director had similar simulation outcomes to the CEO but with two to three character-state failures. The Manufacturing and Quality Director simulation took the most extreme evolutionary trajectory and highlighted the potentially disastrous effects of diversity in decision-making. In future, this approach may also be used to explore underlying opinions, beliefs and attitudes along with their potential consequences on the evolutionary trajectory of a firm when introducing an entirely new manufacturing technology, practice or policy.