When the Measurement Lies

Introduction — When the Numbers Look Precise but the Picture Is Unclear

A flexible-packaging plant completes a major production run. The print looks sound, registration is accurate, color meets specifications, and lamination is complete with no visible abnormalities. Yet the quality laboratory reports a problem: some samples do not meet the required seal strength.

The production manager requests a review of sealing temperature. The process engineer suspects pressure or dwell time. The application-development specialist questions the layers' compatibility. The material supplier is asked to investigate the delivered batch. Production is placed on hold, additional samples are collected, and a root-cause investigation begins.

Then something unexpected happens. The same production batch is tested again, but the results are inconsistent. One specimen passes, another fails, and a further test produces a considerably higher value. Everyone looks at the press, the materials, and the production settings. Almost no one asks whether the measurement process itself could be contributing to the problem.

This is one of the less obvious traps in industrial troubleshooting. A numerical result creates an impression of certainty. When an instrument displays a value to several decimal places, we may treat it as an unquestionable fact. Yet an instrument does not reveal an absolute truth about a product. It produces a result influenced by the instrument, test method, specimen, operator, and environmental conditions.

Sometimes a measurement is highly repeatable but does not represent the product property we intended to evaluate. Sometimes the method is sound, but the samples are biased. Sometimes the instrument is correctly calibrated, but measurement variation is too large to reliably distinguish between an acceptable and a defective product.

An effective investigation therefore begins with three separate questions: Are we measuring the right property? Are we measuring it reliably? And does the result support the decision we intend to make?

This article examines these questions in conventional printing, liquid electrophotography (LEP), packaging applications, and converting. It offers an engineering approach to evaluating measurement reliability before changing the production process.

1. Before Declaring a Product Defective, Define What Was Actually Measured

Industrial quality control depends on comparing a product requirement with a test result. That comparison matters only when the quantity being measured is clearly defined, and the test method genuinely represents it.

Metrology uses the term measurand for the specific quantity intended to be measured. That definition matters more than it might initially seem.

When we say that we tested seal strength, did we measure the maximum force required to open a seal, the average force along the separation path, the package's resistance to internal pressure, or its ability to remain leak-tight? These properties are related, but they are not interchangeable.

A package can exhibit a high opening force and still have a localized leakage path. Conversely, a package may be leak-tight while showing a relatively low opening force and still meet the application requirement, provided that requirement was established in product design.

Similarly, optical density is not the same as color accuracy. A ΔE00 measurement does not necessarily describe print uniformity, fine-detail reproduction, or rub resistance. In adhesion testing, the force needed to separate two layers is only part of the story: the locus and mechanism of failure can be more informative than the numerical result alone.

For this reason, the investigation must start with the product's functional requirement. Only then can you select an appropriate test method. Otherwise, we risk measuring the wrong property with excellent precision.

1. A Calibrated Instrument Is Not Necessarily a Reliable Measurement System

A common assumption is that an instrument with a valid calibration certificate must provide reliable test results. Calibration is essential, but it is not sufficient.

Calibration establishes the relationship between the values an instrument indicates and appropriate reference values with metrological traceability, under defined conditions. It can characterize error and uncertainty, but it does not demonstrate that every future test using that instrument will be fit for its intended purpose.

Other factors matter: resolution, stability over time, operator-to-operator variation, repeatability, specimen preparation, and the test environment.

Consider a force-testing machine that has been properly calibrated. Does that guarantee that specimens of different widths can be compared directly? Does it ensure that changing the crosshead speed will have no effect on a viscoelastic material? Does it guarantee that different operators will position the specimen identically in the grips? No.

The instrument may be functioning correctly while the broader measurement system is unsuitable. The same principle applies to color measurement. A calibrated spectrophotometer does not guarantee comparable results when measurements are made under different instrument conditions, on different backing materials, or on surfaces with different textures.

We must distinguish between instrument conformity and measurement-system fitness for purpose.

1. Where Does Measurement Variation Come From?

When two tests do not produce identical results, the difference may have several sources. The first is genuine variation in the product. Specimens made on the same press can differ in material thickness, ink distribution, coating weight, sealing quality, or microscopic substrate structure. The second source is the measurement process itself.

Measurement System Analysis (MSA) helps determine how much of the observed variation is associated with measurement. A central family of methods is Gage (or Gauge) Repeatability and Reproducibility, commonly written Gage R&R.

Repeatability. Repeatability characterizes the variation observed when a measurand is measured under defined, nominally unchanged conditions—for example, with the same instrument, operator, method, and short time interval. If successive measurements of a stable color patch vary substantially, the measurement process warrants investigation.

Reproducibility. Reproducibility concerns agreement under changed conditions. In industrial Gage R&R studies, operator identity is often the varied condition; other studies may examine instruments or laboratories. Systematic differences between operators may point to specimen preparation, positioning, handling, or interpretation of the method.

Bias. Bias describes a systematic difference between the expected measurement result and an appropriate reference value. An instrument can be remarkably consistent while consistently reporting a value that differs from the reference. Good repeatability, by itself, does not establish trueness.

Stability. A measurement process can be consistent at the beginning of a shift and change later because of temperature, contamination, wear, mechanical movement, or another influence. A one-time verification may not reveal that drift.

Measurement uncertainty. Measurement uncertainty characterizes the dispersion of values that can reasonably be attributed to the measurand based on available information and the measurement model. It is not equivalent to measurement error; nor is it removed by displaying more digits after the decimal point.

Gage R&R can quantify important components of variation, but it is not a complete substitute for evaluating bias, traceability, method effects, and uncertainty. The distinction becomes critical whenever measurements are used to accept or reject material.

1. Case Investigation — A Package That Failed the Seal-Strength Test

Return to the flexible-packaging plant. To illustrate the method, suppose the product specification requires a minimum seal strength of 12 newtons per 15-mm test strip. This value is illustrative, not a general industry requirement.

The initial results are:

These values demonstrate variation but not its source. Differences may arise from genuine specimen-to-specimen variation, the measurement system, or both.

The intuitive response is to inspect the sealing machine. Instead, before changing temperature, pressure, or dwell time, the team audits the test procedure. Several details were not recorded consistently: how strips were cut, whether their width was verified, how they were positioned in the grips, whether they slipped, what extension rate was used, and whether all operators used the same support configuration.

Can these details matter? Yes. ASTM F88/F88M-23 describes methods for measuring the seal strength of flexible barrier materials, including different ways of supporting the test tails. The force measured can include a bending contribution, and specimen support can affect the result. Unsupported, 90° supported, and 180° supported configurations are not automatically interchangeable.

A new hypothesis therefore enters the investigation: part of the apparent production variation may be caused by inconsistent testing. But a method effect does not prove the product is acceptable. Product conformity still has to be established with a specified, validated method appropriate to its end use.

First, verify the method. Specimens must be taken from defined positions, prepared consistently, and tested using one correctly specified configuration. The grips, load-measurement system, alignment, extension speed, calibration status, and observed failure mode should be reviewed.

Next, separate sources of variation. A carefully planned study should estimate the contributions from operators, specimen preparation, and variation within the production batch. If substantial variation remains after testing is controlled, the investigation returns to materials, seal construction, and sealing-machine conditions.

The product may indeed be defective. The test may be creating some of the variability. Both may be true. The objective is not to discredit the laboratory, but to identify how much of the observed variation comes from the product and how much from measurement.

1. The Special Challenge of Destructive Testing

Seal-strength, peel-strength, tear-strength, and many other mechanical tests alter or destroy the specimen. This makes measurement-system studies more difficult.

In non-destructive testing, multiple operators may be able to measure the same stable specimen repeatedly. In a destructive peel or seal test, the layers are separated, and the original condition cannot be recreated at the same location. Different specimens must be used, but those specimens may not be identical.

How can we distinguish instrument variation from operator effects and genuine differences between pieces?

The answer is to design the study around the available specimens and the degree of homogeneity that can reasonably be demonstrated. When a batch can provide sufficiently equivalent specimens, groups can be allocated across operators and repeated test conditions. Depending on the design, a crossed, nested, or expanded model may be appropriate.

In a crossed study, operators evaluate each relevant specimen group; destructive testing requires separate pieces whose equivalence within each group can be justified. In a nested study, operators are assigned within specimen groups rather than crossed with every operator. The choice depends not simply on whether a test is destructive, but on which variance components can be identified given the actual sampling design.

A fundamental limitation remains: if specimen homogeneity cannot be supported, real material variation can become confounded with measurement-system variation. Sophisticated statistical software cannot rescue an experiment whose design cannot separate those effects. The sampling scheme must be improved, or the limitation acknowledged explicitly.

Destructive testing therefore requires both sound laboratory technique and sound experimental design.

1. Color Measurement — When the Instrument Works but Results Change

Color measurement is central to process control and quality release in industrial printing. Spectrophotometers provide spectral and colorimetric information; optical-density and tone-value measurements serve other process-control purposes depending on the technology and control system.

Despite the sophistication of modern instruments, results can be misleading when measurement conditions are not consistently defined. ISO 13655 addresses spectral measurement and colorimetric calculations for graphic arts. Its measurement conditions, including M0, M1, and M2, are particularly relevant when substrates contain optical brightening agents (OBAs).

OBAs may respond differently under different illumination conditions. Therefore, an M1 measurement is not necessarily equivalent to an M0 or M2 measurement. If two suppliers measure the same sample under different conditions, a difference in Lab values cannot automatically be attributed to the printing process.

Backing material. Partially transparent substrates can yield different results over white and black backings. This is especially important with thin films or low-opacity structures. Specify and maintain the backing condition in accordance with the relevant method.

Aperture and location. A nonuniform printed area can produce different readings depending on aperture size and precisely where it is measured. A patch that includes part of a light streak may produce a different value from an adjacent defect-free area. This is not necessarily instrument instability; it may be real spatial variation in the print. Instrument repeatability with the specimen held fixed should therefore be distinguished from repeatability of the complete measurement operation, including specimen repositioning.

Surface condition. Gloss, texture, orientation, and the geometry of reflected light can affect readings according to instrument design and material type. Gloss, semi-matte, and matte surfaces differ not just in appearance, but potentially in their measurement behavior.

These distinctions matter whenever color data are used to justify recalibration. If inputs were collected under inconsistent conditions, a new calibration can respond to a measurement artifact rather than a genuine change in print performance.

1. Offset, Flexography, and LEP — Similar Metrics, Different Mechanisms

The metrological principles are shared across printing technologies, but the mechanisms behind the measured result differ.

Offset — Not Every Density Change Is an Ink-Supply Change

In conventional offset printing, optical-density readings are used, among other purposes, to follow ink-system behavior. If one reading is lower than expected, we must first determine whether it represents the process. Was it taken in the same location and under the same conditions? Is the substrate equivalent? Does the variation run across the sheet or along the production sequence? Could the change reflect slurring, doubling, altered tone reproduction, or some other transfer phenomenon?

Changing ink feed may move the measured number, but it does not necessarily remove the reason the number changed.

Flexography — Spatial Nonuniformity Becomes Measurement Variation

Flexographic transfer is affected by anilox characteristics, printing plate, mounting tape, impression conditions, ink properties, and substrate transport. When a solid area contains banding or nonuniformity, a single OD or Lab reading may not characterize it. Repeat measurements at different positions may show a wide spread while the instrument itself remains stable.

Separate two objectives: characterizing the measurement system and mapping real print nonuniformity. The first calls for controlled specimens and locations; the second calls for a spatial sampling plan along the web's machine direction (MD) and cross direction (CD).

LEP — Calibration, Uniformity, and Product Measurement Are Different Questions

In liquid electrophotography, color control links digital image formation, liquid-ink development, and transfer to the substrate. Color measurements help determine conformity to specified targets but may not fully describe spatial nonuniformity or transfer defects.

An average color deviation points toward one family of checks; a localized or periodic defect points toward another. Where supported by the press platform, dedicated uniformity correction must be distinguished from general color calibration, and both must be used according to their intended purpose and approved procedures.

A measurement approach that does not distinguish these different forms of variation can lead to an inappropriate corrective action.

1. The Hidden Sampling Error — We Measured Correctly, but Sampled the Wrong Product

Even a validated method and a well-characterized instrument can support a wrong conclusion if samples do not represent production.

Imagine a long web-production run with one inspection specimen taken at the beginning and another near the end. Both pass. Does that establish that the entire roll meets specification? Not necessarily. A temporary disturbance in tension, temperature, impression pressure, or ink transfer could have occurred between those sampling points.

For web-fed printing, sampling should consider positions in MD and CD, production time, possible periodicity, and process conditions. In sheet-fed printing, the sampling plan must account for sheet sequence and within-sheet location. When investigating dimensional or mechanical behavior in paper and board, distinguish sheet travel direction from fiber or grain direction.

Sampling needs to follow the suspected failure mechanism. A periodic defect requires sufficient spatial coverage and sampling frequency to capture its repeat distance. Poor sampling can miss it—or create a misleading apparent periodicity. A short-lived event requires time-based coverage. A localized structural fault may require targeted sampling rather than random locations alone.

Inline machine-vision systems can provide continuous or extensive defect mapping, but their performance must also be validated for relevant defect classes, detection sensitivity, and false alarms.

Increasing the number of samples will not necessarily correct a biased sampling design. Sometimes the most important question is not how many specimens were tested, but why those particular specimens were selected.

1. When the Reading Is Near the Specification Limit

Many factories make conformity decisions by comparing a reading directly with a tolerance boundary: above the minimum is a pass; below it is a fail. This is simple, but it does not always account for measurement uncertainty.

Suppose the minimum seal strength is 12 N/15 mm and the observed result is 12.2 N/15 mm. Can we be certain that the product conforms? The answer depends on the relevant uncertainty, test method, and decision rule.

For illustration only, assume an expanded measurement uncertainty of 0.7 N/15 mm for that result. It would be inappropriate to interpret 12.2 as an exact value with no uncertainty. This does not automatically mean the product fails. It means that conformity assessment must account for an agreed approach to risk.

ILAC G8 discusses decision rules and how measurement uncertainty may be incorporated into a statement of conformity. One option is a guard band, a margin between the specification limit and the acceptance threshold. If a conservative rule were agreed that accepts only when the measured value minus expanded uncertainty meets the stated minimum, the illustrative acceptance threshold would be 12.7 N/15 mm.

That threshold is not an industry-wide requirement; it is one possible decision rule. The appropriate rule must follow contractual requirements, applicable standards, product risk, and the agreed balance of customer and producer risk.

The underlying mistakes are false acceptance—accepting a product that does not actually conform—and false rejection—rejecting a product that does. The first may cause failures in use, returns, and safety risks. The second creates avoidable waste, downtime, cost, and unnecessary process changes.

A defensible quality decision therefore requires more than reading a number. It requires understanding measurement reliability and the risk of being wrong.

1. How to Design a Practical MSA Study in a Printing Plant

An MSA project need not begin as a large statistical exercise. It should begin with the decision the measurement is expected to support.

First, define the response: ΔE00, optical density, seal strength, peel strength, coating thickness, die-cut dimensions, or another relevant property. Define the instrument, method, environmental conditions, specimen preparation, and possible operator effects.

For suitable non-destructive measurements, a crossed study can be arranged in which several operators measure multiple stable specimens repeatedly. An illustrative starting design might use ten specimens spanning the actual operating range, three operators, and three repetitions. These numbers are not universal requirements: the study size must match the process, expected variation, and analytical purpose.

Randomize or balance the measurement order where feasible to reduce time trends, anticipation, and order effects. Samples must represent the decision-relevant range. If specimens are almost identical, the study may provide limited information about the system's discriminatory capability across the required range.

For destructive tests, select a study structure based on justified specimen equivalence and an appropriate statistical model. Analyze the identifiable contributions of instrument, operator, specimen, and interactions.

Do not stop at a single percent Gage R&R number. Examine resolution, bias, stability, traceability, and uncertainty in relation to the actual specification and intended decision. Familiar percentage thresholds can be useful screening conventions, but do not adopt them blindly without considering process variation, tolerances, intended use, and the consequences of a wrong decision.

A system acceptable for trend monitoring is not necessarily adequate for a pass/fail decision extremely close to a specification boundary.

1. Troubleshooting Protocol — When the Measurement System Becomes a Suspect

When results are inconsistent, a staged investigation can prevent unnecessary changes to production equipment.

Sequence matters. Improving instrument repeatability will not compensate for specimens that fail to represent production. An excellent Gage R&R result will not make a fundamentally unsuitable test method relevant to the product's function. And if measurement is proven adequate, a genuinely unstable production process still needs to be investigated.

The goal is to locate the variation and show that the corrective action improves the reliability of decisions based on the data.

1. Reliable Measurement Is a Prerequisite for Reliable Experimentation

Root-cause investigations often use Design of Experiments (DOE) to identify influential factors and their interactions. Every conclusion from such work depends on the quality of the response data.

If a measurement system cannot resolve a small but operationally important change, an experiment may report no difference even when a real effect exists. Conversely, a noisy or drifting measurement system can make an unimportant variable look influential.

Suppose an experiment compares two approved surface-treatment conditions intended to improve ink adhesion to a synthetic substrate. The resulting test values differ slightly. Does that difference justify changing the production process? Not until the precision and variation of the test method—and real specimen-to-specimen variation—are understood.

If the observed effect is small relative to relevant measurement uncertainty and material variability, the evidence may be insufficient to establish that one condition is better. Measurement-system evaluation belongs in the experiment plan, not only in the post-mortem when the results are confusing.

The same principle applies to Statistical Process Control (SPC). A control chart can detect changes in the observed data stream, but unstable measurement bias or excessive measurement noise may conceal real process changes or create artificial signals.

Before improving process stability, we need confidence that the measurement process is not hiding instability—or inventing it.

1. From Better Measurements to Better Decisions — A Shared Responsibility

Measurement reliability is not the quality laboratory's sole responsibility. Operators help ensure consistent sampling, recognize unusual behavior, and maintain controlled production conditions. Production managers must know when a result warrants a process change and when a verification test is needed first.

Quality engineers match test methods to requirements, train personnel, design sampling plans, and validate measurement capability. Application-development engineers must ensure the measured property reflects product performance under real-world use conditions. Color specialists must distinguish genuine colorimetric variation, spatial print variation, and variation introduced by how the measurement is made.

When these functions work in isolation, departments may reach apparently sound conclusions from data that do not describe the same phenomenon. When they agree on the measurand, method, conditions, sampling design, and decision rule, they build a quality system that produces consistent, defensible decisions.

The objective is not to generate more numbers. It is to generate information you can trust.

Conclusion — Before Fixing Production, Make Sure You Can Trust the Measurement

Industrial printers invest enormous effort in stabilizing color, improving image uniformity, securing ink adhesion, controlling converting accuracy, and qualifying end-use durability. Advanced instruments, analytical software, and international standards support those efforts. But the more we depend on data, the more important it becomes to understand how those data were created.

A calibrated instrument does not guarantee a suitable method. A suitable method does not guarantee representative sampling. And representative sampling does not make every reading near a specification limit an unambiguous basis for acceptance or rejection.

Measurement reliability is not a single instrument characteristic. It emerges from the entire system: a well-defined measurand, appropriate equipment, a validated method, trained operators, representative specimens, and sound engineering interpretation.

The professional approach is not to distrust every result. It is to know when and how to evaluate a result's reliability. When a product fails a test, investigate the product and process. But when readings are inconsistent, contradict observed product performance, or differ substantially across operators or laboratories, be prepared to investigate the measurement system too.

Sometimes the product is genuinely defective. Sometimes the test is unsuitable or inconsistent. Sometimes both are contributing.

The ability to tell the difference is fundamental to engineering quality.

Before declaring that a product fails its requirements, make sure the measurement system can tell you so reliably.

Technical References

1. NIST/SEMATECH. Engineering Statistics Handbook, Chapter 2: Measurement Process Characterization.
2. NIST. Gage R&R Studies.
3. ASTM F88/F88M-23. Standard Test Method for Seal Strength of Flexible Barrier Materials.
4. ISO 13655:2017. Graphic Technology — Spectral Measurement and Colorimetric Computation for Graphic Arts Images.
5. ISO/TC 130. Guidelines for Using Print Production Standards, 2024.
6. ILAC G8:09/2019. Guidelines on Decision Rules and Statements of Conformity.
7. ILAC G17:01/2021. Guidelines for Measurement Uncertainty in Testing.
8. Minitab. Gage Studies with Destructive Testing.