Welcome to Production Hell
Back in the middle of 2017, as Tesla was ramping up the production of the Model-3 and the company was worth only $63.5 billion (as opposed to $1 trillion today), Elon Musk famously stated that he was entering “Production Hell“. Evidently, he made it through just fine.
But people reading this might think; “Isn’t Musk exaggerating? How bad can it be, really?”
Let me tell you, it can be pretty friggin bad…
I was working on a turn-around engagement at a distressed company earlier this year. Aside from their not knowing how to hire and build talent (instead, acquiring bodies to fill holes) and not paying attention to the sudden rise in their raw materials (and selling product at well below what it cost them to produce); their on-time deliveries were less than 20% and their scrap and rework exceeded 10% of revenue.
Indeed, their manufacturing floor was production hell. It was a place of perpetual torment filled with lost souls who were weeping and wailing. All that was missing was the fire and brimstone for it to truly be Hell on Earth.
When I arrived, watching the activity on the production floor reminded me of watching very young children playing soccer for the first time. Nobody was playing their positions, everybody was playing the ball; except most of the time they didn’t know where the ball was either.
The production process should flow like a river that moves at a constant and predictable rate without pause; nice and smooth. The flow at this manufacturer was more like driving into pre-COVID Manhattan at 8:30 on a Monday morning; stand stills mixed with fits and spurts of forward movement, zig-zagging in and out of lanes, with the occasional fender-bender and obligatory expletives.
The challenges that are being shared below are not anecdotal or one-offs; they were pervasive daily occurrences as if this was the way the business was designed to operate. And, although it might look like these challenges are cellularized, they are interconnected with each effecting the other (oftentimes amplifying the detrimental effects).
I am not placing fault with the employees who were largely left adrift without guidance, support, or encouragement. Nor am I placing fault with management who (in addition to being untethered and sometimes under-qualified) were not free to do their job without meddling from the owner of the company. So that leaves the owner…
As for me, I was only there two months; and even eggs take two minutes. But in those two months I was able to extend their runway and give them a fighting chance.
Production
People didn’t know what to build. Every morning there would be a status meeting where the production management and supervisors would review the previous day’s production numbers. They would mostly concentrate on productive hours (hours spent producing product) per department and what product was produced. They didn’t discuss failures such as scrap and rework except when the productive hours were lower than expected. And due to the lack of discipline in capturing production data, it was impossible to tell if they were winning or losing; except at the end of the month when they didn’t make any of their numbers.
Afterwards, there were “negotiations” between sales and production where production would share what the day’s and week’s production plan was and sales would share the demands of the customers (and with less than 20% on-time delivery, there were plenty of customer demands). They would go back and forth until the lies they mutually settled upon satisfied everyone in the meeting (but still were not going to happen).
However, regardless of the settled plans, the production employees tended to build what they could see could be built rather than what should be built. And since the KPI was set to productive hours, and the time spent searching for what should be built was not productive, the behavior induced was to build whatever they could. And if they ran out of things they could see, they would just build whatever they could to maintain their productive numbers. Even building scrap was considered productive time.
Countermeasures: Most of the parts for a build moved through the shop on skids with one build per skid. To be better able to see the flow and what was ready to progress through the production floor, they would put a “green cone” on the skid to indicate the materials on it were ready for the next operation and then made sure the skid was placed near where the next operation would occur. This helped the operators see what was next in the queue and where it was.
They had an ERP system, but there was a lack of discipline in its use. The operators would routinely not log into the right job or log in at all. And they would often forget to log off a job. Every day, the department supervisor would have to manually adjust the time. Where the adjustments accurate? Who knows; there was no way of telling. Of course, all this made variance reporting impossible. But this was okay since they never compared estimates to actuals.
People didn’t know how to build. When it came time to produce, the “how do we build this” was largely a combination of tribal knowledge and guesswork. Other than the information on the routing (they called it methods), which was sparse, the employee was largely left on their own. There hardly existed any set-up instructions or production documentation. When I asked about work instructions, the owner called-up the PDF of the ”owner’s manual” associated with the equipment; hardly what I would call work instructions. And certainly this was nearly worthless to the operator.
As detailed in my previous article, “They are dumb”, when a new employee was hired, there was no formal on-boarding process other than “welcome to the company”. There was no “welcome to your department”, “welcome to your workcenter”, or “welcome to your machine”. The lack of processes naturally meant that there was no process owner; or is it that, since there was no process owner, there were no processes? Chicken and the egg?
There were no formal training programs in place. There was no person who was deemed certified to train on a machine or operation (and no standards or criteria for how “certified” might be established). And there were no mentorship programs to bring new people up to task. There wasn’t even a program in place to ensure the operators knew how to read the prints of the parts they were making.
Adding to the already overwhelming challenge for those in production was the poor state of the engineering drawings. Although the Engineering Manager was a very capable person, he was relatively new to the position and he had inherited decades of drawings that were deficient in one way or another; some were missing details (or the details were just wrong) such as for hardware call-outs and dimensions, some were presented in an atypical fashion which led to the operator becoming confused (often bending things backwards as a result).
Countermeasures: To the Engineering Manager’s credit, he did put in place a program of “red-lining” prints (later expanded to routings). The operator, upon seeing an error, would red-line the error and send it to engineering for correction. An engineer would then make the correction and close the loop by informing the person who reported the error that it had been corrected. However, the process was not fully mature as the errant prints would sometimes remain available either as hard-copy or someplace in the computer and the mistake would reoccur. This led to suspicions that the mistakes had not been corrected when, in reality, the mistakes were never fully expunged from what was already in-play.
It also didn’t help that the company went without a Quality Manager for 18 months, having just hired one at the end of my engagement there. What quality processes they did have in place were porous and not reliably followed with many defects slipping through. Customers would make a special demand that a specific inspection operation be implemented to catch the more chronic or critical defects. This added to the production cost and would not be necessary if proper work instructions and drawings existed.
They did have a “tag” system for quality control in place. The tag would have the details of the routing (methods) with each operation identified. The operator, having completed an operation successfully, would check-off the operation and put their initials. A casual inspection of these tags at final assembly would show all the operations checked-off and initialed – by the same person using the same pen. This told me that the tag system was worthless as a quality control.
Once, while in a meeting on the production floor, we discussed a situation where 80 parts were built wrong and were scrapped. When asked if there was any first-piece inspection or in-process inspection, the answer was yes. It was obvious that this was untrue; that neither a first-piece inspection nor an in-process inspection existed – or if it did exist, it was not being done.
And the Continuous Improvement efforts were merely fire-fighting exercises. Some chaos or catastrophe would happen and the person responsible for fighting the fire would come out and work at putting out the fire; but he did nothing to build problem solving skills in the production team or foster a culture of leadership. In fact, his attitude was too often that of being annoyed and having contempt for the people who had the problem.
Countermeasures: A “SEAL Team” was formed from the senior leadership in the company with a senior leader being assigned to each department. They were to look for opportunities for improvement and share the opportunities for improvement with the other members of the SEAL Team at the end of the day when they would be reviewed and an action-plan put in place (in essence, a Kaizen Event without calling it that). Improvements that were going to take a longer period of time or more effort were dropped from the SEAL Team efforts and converted into projects. Although the owner of the company participated at the beginning of the effort, he dropped out after a few days and others had to take-up the slack.
Inventory Management
People didn’t know where to find the inventory they needed or where to put what they built.
Although putting green cones helped to see the flow of product on the floor and whether it was ready for the next operation or not, it did not tell you what was on the skid. So, if the skid was not moved from the previous operation to the area for the next operation, it would still be difficult to find. There were tags on the skids showing what was on it, but this took time.
Countermeasures: The building was approximately 250,000 square feet in size. There were several rows of vertical beams spaced evenly apart that supported the roof. Barcode “license plates” were created and placed on the vertical beams and for the material on the skids. As the skids moved through production, the material handler would use a long-range scanner and scan the license plate on the skid and the nearest license plate on the vertical beam to check the material into that location. The operator would then be able to see where the material they needed was. In this way, the inventory locations were dynamic.
Most parts and hardware inventory was managed through tribal knowledge. And while the scanners worked well for larger inventory items on skids, the approach did not work as well for small hardware items. On a daily basis, it would happen that several labor-hours would be spent looking for a part that cost pennies. The costs in productivity and delayed shipments because of not being able to find these items was probably in the tens’ of thousands of dollars per month. However, the failures here were largely due to a lack of discipline (or training) on the part of the people in production combined with opportunities for improvement in the processes themselves.
Planning and Scheduling
Planning and Scheduling was more guesswork than one would expect. The company didn’t have a satisfactory handle on how long it took to build anything they built. They had routings (methods) for how a product was expected to be built with estimated times, but they did not compare actuals to estimateds and they did not analyze their variances. This, combined with scrap and rework, meant that the company was never able to really plan, but was rather chronically scheduling and rescheduling.
The planner/scheduler was quite capable. But he was planning and scheduling using time buckets that were proved to be inaccurate and this led to over and under scheduling. Even a schedule created in the morning for that day would be wildly inaccurate by the end of the day. To remedy, they really need to track actuals to estimates to validate what they believe is occurring.
And it didn’t help that sales was reprioritizing the production schedule on almost a daily basis. While this might be a workable approach if there is enough slack in the system to support it (which is a different kind of problem), it is catastrophic otherwise. You could actually see the compression and expansion on the floor from the engineering tower in the middle of the facility; it looked like a “slinky” or worm making its way through the production floor.
And these constant changes to the schedule did not just effect production, but also procurement; with the result being product was delivered either too early or too late – or not ordered at all – far too often.
Wrap-up:
The company has been in business for over thirty years and always owned by the same owner. Yet this is unmistakably a very sick and broken company. There is no function of the company that was operating at an acceptable level; far from it. It took a very long time for the company to get this screwed up and it is going to take a long time to fix it. And there is no pill to take or button to push that will make it easier. It’s going to take a lot of hard work by talented and capable people.
And although their losses were 10% of revenue month-on-month and they were pushed to the financial brink, my having them reprice their products they sold so that they were not selling at less than their cost bought them some much-needed runway. They will live to continue the fight, but not yet experiencing the enjoyment of being alive.
Their biggest challenges now will be; i) having highly skilled talent in key positions (and the owner allowing them to use that talent), ii) not trying to do too much too fast (with the risk being the entire organization spins out of control), but rather being incremental and methodical, iii) and being patient but pressing (a skill the present owner lacks).
As they say, the way to eat an elephant is one bite at a time. It’s dinnertime.
A “thank-you” to Don Burshnick who was my wingman on this project, who validated the facts and content as presented here, and contributed his insights in the creation of this article.
About the author
Joseph Paris is an international expert in the field of Operational Excellence, organizational design, strategy development and deployment, and helping companies become high-performance organizations. His vehicles for change include being the Founder of; the XONITEK Group of Companies; the Operational Excellence Society; and the Readiness Institute.
He is a sought-after speaker and lecturer and his book, “State of Readiness” has been endorsed by senior leaders at some of the most respected companies in the world.
Click here to learn more about Joseph Paris or connect with him on LinkedIn.
