Analytics header

Showing posts with label KPI. Show all posts
Showing posts with label KPI. Show all posts

Friday, May 18, 2012

How hard is it to define MI requirements?

Well let's say its not straightforward! For quite some time I have been trying to explain the difficulties in providing clear specification for Manufacturing Intelligence (MI) system. In addition the life science industry is still bound by traditional methods of analysis, requirements specs and functional specs that simply put will not work for MI. MI's whole purpose is to provide information to somebody to support his creative behavior as he explore root-causes and solutions in the dynamic world of manufacturing operations. (This is an extract from a forthcoming article in Pharmaceutical Engineering).

The right approach is based on the critical element of understanding how people use information to solve problems and gauge performance. There is a clear need to provide effective and relevant information necessary to support the information consumed by the different roles in the manufacturing operations. Identifying what needs to be measured is a fundamental principle but it is not sufficient, the information also has to be arranged in a usable manner. Therefore it is important to study and understand information consumption patterns by roles. Take for example the information consumption pattern for a supervisor in a biotech plant that is creatively analyzing a production event.


The production supervisor glances at his dashboard and observes that the Cell Density is not within acceptable limits. He immediately navigates to view the “Cell Density by Time” trend over the last 2 weeks and observes a negative trend beginning around “Mon.” that indicates something is seriously not in order. 
To begin the analysis, he examines the “Media Batch Feed Schedule” to see if there is any correlation between the trend and the Media that is being fed to the bio-reactor. This action is obviously based on intuition possibly because he has seen that before. Seeing that there is a correlation between the change to a new Media batch feed when the trend start he decides to take a look at the “Exception By Batch” information and notices that this specific batch had an unusual number of exceptions. He then dives deeper into the data by analyzing an exception Pareto for the suspect batch. He finds a high number of operator errors, which clearly highlights the root cause of the trend. Finally since he is accountable for operational profits he decides to take a look at the cost impact of this event in order to understand what the impact is to the plants financial performance (see graphic below). Unfortunately the cost impact is substantial and thus he as to take action to mitigate this increased cost. 


The scenario shows the power of “Actionable Intelligence”. The supervisor has all the information he needs in order to quickly and effectively analyze the situation to determine root-cause and he can take action based on the results. The path that the supervisor decides to take in the example above is one of several that could have been used to detect and diagnose the Cell Density performance issue. It is this type of self-guided or self-serve analysis that really shows how information is consumed to meet a specific goal and should be the common pattern for the information required by a specific person or role. These requirements have direct bearing on the information context and data structures that must be provided, and the dimensions by which the metric is analyzed or “sliced and diced”. Although this seems trivial at first the requirements that this analysis patterns has on the underlying information and data structures is significant and is a critical component of the system design. It is not enough just to collect the data; it has to be arranged in a manner that enables this unique type of analytic information consumption.

Thursday, December 29, 2011

Why Do We Still Use Spreadsheets?

Sometimes I find some unfinished article while cleaning up – something I should obviously do more regularly! So here is one of these excerpts that I found particularly relevant as I am discussing the topic of “Manufacturing Intelligence” with a number of companies.

Manufacturing systems software vendors continuously tell us that you cannot have visibility into your operations without a software application, which I have to agree is generally true. This forces us to sift through the onslaught of offerings full of buzz words such as “metrics”, “digital dashboards”, and “business intelligence platforms”. Yet, it is remarkable that one of the most commonly used tools to capture and manage information from the shop floor is The Spreadsheet - typically Microsoft’s Excel. In some cases, even with a major ERP system investment, the Spreadsheet is still the primary source of timely data collection about the manufacturing operations. In other cases, expensive solutions are put in place to capture and collect data from automation equipment but fail to provide the information in a useable context and once again users resort to the spreadsheet.


Why is it then that manufacturing organizations resort to solutions that are based on a spreadsheet? It is typically not because of lack of understanding about information systems or the skills required to use them. It is because a spreadsheet provides the flexibility and ability to manage and present shop floor information in the most useable and advantageous manner. (By the way the common term for “manage and use” is “information consumption”.) Remember that a manufacturing manager’s main focus is productivity and quality. They use this information to obtain metrics about the value stream that they are trying to manage because they need to know how they are performing in real time. This need is similar to that of a sport’s team, where you know where you stand at every second of the game. You don’t have to wait until tomorrow morning’s newspaper to know who won the game. Running a manufacturing operation without real time metrics is like bowling without being able to see the pins. You can see some of the action, you know that something happened, but you don’t know what the result was.

Of course in recent years, manufacturers have gained some visibility with the increased application of technology, but they are still far from what is possible. I also believe that most of the vendors are clearly aware of the needs and I hope that they we will soon start to see Manufacturing Intelligence applications with the flexibility and convenience that we really need.

Wednesday, November 2, 2011

A Foundation For Quality (and performance...)


Next week is ISPE’s annual meeting in Texas and once again I am participating in an interesting session titled “Operational Excellence - A Foundation for Quality”. We held a similar session last year and it was a great success with more than 90 people participating. The topic that I am covering this year is about the current capabilities of automation technology andinformation systems that provide a vital ingredient in enabling operational excellence. I will be joined by speakers from Pfizer, Celgene, Amway and of course NNE Pharmaplan.

My co-presenters will share experiences about OperationalExcellence, Quality by Design (QbD), and Process Understanding. All-in-all an interesting combination of topics that may initially seem loosely related but they are in fact deeply related. They stem from the some old notion that manufacturing performance is a holistic concept. I have mentioned in a previous post that we are “re-inventing the CIM wheel” but at the same time we are also looking at it in a new perspective with much more modern and usable technologies.  I strongly believe that before we venture with new ideas we have to look and learn from the past, it is very likely that somebody has had a similar problem and may even have a solution. So CIM, PAT, QbD, DFM it really is all related, related to trying to excel at we do and work more synergistically to accomplish our manufacturing goals. 

Thursday, October 13, 2011

Metrics & Performance Management - Again

Not that again! Well it still is a very interesting topic an one that I encounter every time I talk to companies in the life science industries. It seems that there is an increased understanding across all walks of life in the manufacturing organizations that one of the main advantages of system is the ability to gain a better understanding of both process and operations. I hear people asking for metrics, and in the context of gauging their performance. I spend a lot of time trying to understand this trend, and I have no conclusion yet. It may be the economic climate, maturaity of the technology, or maybe the momuntum that operational excellence intiatives have gained?

The membership of the ogranziations that I work with such as MESA and ISPE have also shown great interest in this topic and I am participating in a few events that highlight them. On October 27th we have a MESA webcast about "Harnessing the Power of Metrics" and on November 7th we are organizing a special session at the ISPE annual meeting about "Operational Excellence - A Foundation for Quality". The session will be hosted by NNE Pharmaplan and will include speakers from Amway, Celgene and Pfizer. I hope to have some great discussions, feel free to track me down if you are in attendance.

Friday, August 20, 2010

EMI or BI - what are those anyways?

Lately I have had inquiries from a couple of customers about providing “intelligence”, as in manufacturing intelligence, well I believe that is a good topic I said… (Hint, see my blog’s title).

In most cases their curiosity came from either seeing or working with an EMI or Enterprise Manufacturing Intelligence system. However when I asked about how they would like to use the intelligence I found out that they where asking about BI – Business Intelligence behavior, yet they wanted manufacturing metrics like the EMI system’s demo. That is an interesting discussion that I have mentioned a few times in this blog. I believe that what they really want are timely and sometimes real-time intelligence about their manufacturing operations provided in KPI lists, metrics and charts but also analytics capabilities such as “slicing and dicing” and “drill downs”.

Unfortunately I have found that such solutions are rare if they exist at all. It seems that EMI is a separate animal than BI and the problems that they are trying to address are more complicated than they generally convey in their sales and marketing information. Most EMIs do not provide an effective way to aggregate and correlate information to support BI style analytics, while the BI vendors’ understanding of 'real-time' and timeliness leave much to be desired for application in the manufacturing operations world.

Tuesday, February 23, 2010

What is Intelligence, and why do we need it?

I believe that it is time to get back on topic in this blog, which is Intelligence in manufacturing and manufacturing systems in general. With that in mind I was looking thru my archives and came across something I once wrote as a positioning statement for an intelligence product – I guess it is not hard to figure out what company that was for? So here goes…

In one of my previous posts I tried to bring up the point that we need to consider metrics in the context of what they are needed for and how they are going to be used. I believe that is the best way to understand how to provide the right intelligence in a given scenario. But what is Intelligence? Well that is a very serious subject, but let’s take in the context of manufacturing and process improvement.

Intelligence implies the ability to comprehend; to understand and profit from experience. As such Intelligence is information valued for its timeliness and relevance rather than its detail or accuracy in contrast with "data" which typically refers to precise or particular information, or "fact".

In the context of manufacturing, Intelligence is a fundamental ingredient influencing the system’s level of performance in reaching its objectives. A manufacturing business system (humans included) is a system that learns during its existence. In other words, it learns, for each situation, which response permits it to reach its objectives. It continually acts and by acting reaches its objectives more often than pure chance would indicate. We can observe the following about Intelligence in manufacturing:
Intelligent manufacturing is not a smarter way of producing things; it is a human centric approach where humans interact with the process be it automatic or manual, gathering the right information to take intelligence decisions based on actionable information. It is much more than visibility. Just having the information is of course helpful, but it needs to be taken one step farther. It needs to be provided in a way that people can intuitively capitalize on it using their knowledge and understanding to make effective decisions.
Henri Poincare once noted in a related topic that “Science is facts; just as houses are made of stones, so is science made of facts; but a pile of stones is not a house and a collection of facts is not necessarily science.”
What is an effective decision then? It is a decision that has an outcome that drives increased performance and continuous improvement. Intelligence is therefore not solely about metrics, KPIs or the ability to drill down into the data. In order to increase performance we need to quantify what is important. Hence intelligence is about quantifying what is important, or quantifying the unquantifiable.

Monday, February 15, 2010

Using metrics and what it tells us about Manufacturing Intelligence

In continuation to some of the other posts in the topic of metrics and KPIs, this time I would like to discuss the use of metrics. So, not so much what the metrics are or which metrics are most important, but how do we use them. If we look at how analysis is performed we may be able to gain some insight into how the data should be collected and structured. This may be for example to support lean or process improvement initiatives. The idea regardless of methodology requires that the team or person gain an understanding of the process and its behaviour based on the data at hand.

I think the best way to frame this up is to tell the story of a persona in a simple scenario. Let’s consider Patrick Process Engineer who is trying to understand yield fluctuations specifically and maybe the yield’s behaviour in general. He is perplexed about why he cannot accurately predict yield given that most of the processes are “in control” (yes, another one of these myths about processes). What he really is striving for is an understanding the process and the best way to investigate it – in other words analyze the process.

Obviously Patrick will be looking at the Yield metric(s) and once he sees some fluctuation he will embark on an analysis. Typically he will perform what I call a high-level analysis, which is kind of a “look-around” in the data and maybe other related metrics to determine patterns. But wait, why patterns. Well whether we like it or not when we analyze processes we naturally look for patterns since that is what complex processes really exhibit. I guess I need to write a bit more about the complex adaptive behavior of manufacturing systems, but let me leave that for a future post. For now let’s just say that the patterns really tell us about the dependencies between the different metrics and parameters in the system.

OK, once Patrick has performed is “high-level” analysis he may then start digging a bit deeper in to some of the areas that may lead him to the root-cause of the fluctuation. He may perform some more detailed analysis, maybe choose to monitor some specific metrics, define some additional more detail or focused metrics. If he really is trying some advanced analysis he may try and observe dependencies between metrics for a period of time. That means monitor maybe a few metrics and how they change in relation to each other.

What is described here is really a simple scenario of human behavior exhibited when we perform troubleshooting. Although this as simple behavior for us, when we think about the data and data structures that we need to support what Patrick is trying to do, we may quickly realize that the complexity abounds. Modern business intelligence concepts such as multidimensional analysis, OLAP, and practices for data aggregation provide the tools. However it takes a lot of work and process knowledge to transform the base process data to such information structures.

This I believe is the main challenge in modern manufacturing systems. Compounding this problem is that effective analysis needs to use information from all the host of system that may reside in a manufacturing business, i.e. ERP, MES, Automation, Historians, LIMS and others. Most manufacturing intelligence tools that are in the market today simply do not address this simple scenario. If we do our inbound marketing work correctly and draw up the scenarios above, with Patrick our main persona, the problem definition is straightforward; however the engineering task required to solve it is not.

Monday, February 8, 2010

When is enough, enough? The art of counting jelly beans.

I was reading an article in National Geographic Blog about counting jelly beans that in a way boiled down what my general belief about metrics (see also another relevant post about metrics). Also, I have lately been involved in MESA’s metrics working group and all of this had me thinking…

The current practice in regards to metrics in operational environments especially in manufacturing is the typical engineering tactic - we need to break down the complexity of the system and figure out what metrics we need. Well one approach is to take a look at what other companies are doing – as in MESA paper we try to figure out if there are any best practices. But is this the right approach for any company or for you? I say no! Metrics are a way to represent patterns in the complex systems that enhance our understanding of its dynamics. We can look at one metric and say we are doing good or bad, which works well if you are a computer since we can take a decision based on this metric. Yet unlike computers, we humans are much better at using our knowledge and understanding to detect patterns and react to them. So if you give us a set of metrics that help us interpret what the system is doing then we do much better. That is exactly what the term "Actionable Intelligence" means.

There are a few important factors that I believe are important:

  • A few metrics used independently are never going to give you the true picture of what is going on.
  • Simply copying what other companies are doing will not work either, since metrics should be driven by what you are trying to achieve – and that changes with time.
  • The Accuracy of the metrics are not necessarily important. What! Yes, I just said it; accuracy is not that important. Let’s think about a jelly bean counting competition, you really don’t needs to know the exact number but be the one that is closest to the precise number, in order to win. Michael Schumacher (if you do not know, he was a 7 time Formula 1 GP winner) once said “… you don’t have to drive your fastest to win, but faster than the guy behind you”.
So why do we obsess about having so much data, detail and precision? Why are we always trying to quantify the unquantifiable, as in 6-Sigma. Maybe it is our upbringing as engineers, that constant quest for details. Really all we need is an indication when things will go wrong and when they are going well? This is of course not to say that sometimes we do have to be very precise as well as accurate, such as in engineering tasks, designs, etc. The art of it all is to know when we are precise enough.

In a blog post Stephen Few states “most poor decisions are caused by lack of understanding, not lack of data”. This is exactly my point, we spend so much time working on detail. For example, how much the ocean water level will rise over the next period of time. Some say a few meters some say 30 meters in the next 100 years. But the point here is that even in a moderate scenario calculation; Venice, New Orleans and the SF Bay Area delta, among other places, will be under water. So if you live there – guess what? You are in trouble, regardless of accuracy.