Image Interpretation In The Era Of Super-Resolution Microscopy

Introduction

One of the most significant paradigm shifts in science occurred in the 17th century with the invention of the microscope. The new instrument allowed the observation of minute structures present in common objects, which otherwise were hidden from human vision. The controversy took place among philosophers and scientists at that time: were those structures real and, if so, how did they relate to the known macroscopic world?

Initial skepticism and disbelief were gradually phased out by the progress of scientific rigorousness, to the point where the microscope became an essential observational tool in all natural sciences. Over the last two decades, combined developments in optics, fluorescent markers, and computational methods contributed to the foundation of a new set of optical microscopy techniques grouped with the name of super-resolution fluorescent microscopy (SRM). This new discipline circumvents a practical limitation of optical microscopes, allowing for visualization of biological structures and processes at a nanometer scale, including live cell imaging. The extra level of information offered by SRM has generated high expectations in the scientific community, for the potential contribution to the understanding of molecular biology and the discovery and treatment of diseases. In addition, in 2014 The Royal Swedish Academy of Sciences awarded the Nobel Prize in Chemistry to the three main inventors of SRM, drawing the attention of biologists to these new microscopy methods. Nevertheless, the SRM techniques are difficult to implement (as compared to traditional microscopy methods) and susceptible to imaging artifacts that can lead to misinterpretation of the visual information. In my Ph. D. project, I focus on SRM of tissue sections. Hence, in this essay, I propose that the original questions from the 17th century still hold: what do we really see with SRM? In the sections that follow, I will reflect upon some philosophical and ethical considerations related to SRM. First, I will look into the general concepts of microscopic observation and subjective interpretation. I will then analyze how these can breach scientific integrity in relation to the existing ethical guidelines for image manipulation. Finally, will I will reflect upon the importance of an active debate of these topics among newcomers to the discipline of SRM.

Observation and perception

Both observation and perception are interlinked concepts extensively studied in philosophy of science. Observation is a perceptual process that relates to the acquisition of information involving one or more human senses (e. g. hearing, sight, smell, taste, or touch). According to Bogen, by attending to details of the perceptual experience, observers are able to “obtain useful perceptual evidence simply by noticing what’s going on around them”. Philosophers and scientists from the 20th century have long argued about how this subjective factor of ‘perception’ influences science. For example, Hanson, in his book ‘Patterns of Discovery’, states that “seeing is an experience. A retinal reaction is only a physical state – a photochemical excitation. People, not their eyes, see”. In his later work ‘Perception and Discovery’, Hanson further develops the idea that “what we see are the changes in our rods and cones. Everything above and beyond that is an intellectual construction on our parts”.

In the same line of thought, Johansson and Lynøe propose that the structure of the observations are influenced by the theories and concepts previously acquired by the observers, in what is called theory-laden observations. To the non-initiated in fluorescent microscopy, the image is merely perceived as a composition of red lines crossing in all directions over a black background, whereas for the trained cell biologist, the image contains morphological information of the actin filaments present in the cells. Yet there is more to add. It happens that persons embracing the same knowledge, methods, and principles may report different observations of the same phenomena due to inter-observer variance. Analogously, a given person may report different observations of the same phenomena over time due to intra-observer variance.

Microscopic observation and subjective interpretationThe progress in optics along the 17th century contributed to the invention of the microscope and the telescope. The microscope faced skepticism and disbelief about the true meaning of their representations. The minute structures under study had never seen before and, therefore, were difficult to relate them to the ‘known’ macroscopic world. The telescope proved to be less problematic. Scientists could first observe a distant object (e. g. a mountain), and then visit such a place to corroborate their observations. A similar approach, however, would not work for microscopy for obvious reasons. It was neither possible to physically stretch out the microscopic structures to prove the observations, nor it was possible to compress someone’s body to observe directly in the micro-dimension. Then, how do we relate to the observations of the microscopic world? The following maieutic questionnaire illustrates the complexity of interpreting microscopic images.

Q. What do we see through a microscope? A. The microscope renders a magnified representation of the miniature world in multiple levels, according to the grade of magnification and resolution.

Q. What does this representation mean? A. Such representation does not mean anything by itself but the observer who interprets it and makes sense of it.

Q. How close to reality are the microscopic representations? A. Microscopic observation of biological specimens often requires unnatural preparation by means of fixation or immobilization, slicing, and staining. The microscope must be adjusted to compensate for optical distortions (e. g. chromatic and spherical aberrations), and the imaging parameters (e. g. exposure time, light intensity) need to be optimized for adequate acquisition. All the steps can introduce distortions in the image in the form of artifacts. To this point, as Hacking suggests: “the images are ‘true’ only when one [the microscopist] has learned to put aside the distortion”.

Q. How do we know if what we see is real? A. A commonly accepted method for validation is by observing the sample under different physical methods (e. g. optical microscope vs. electron microscope). If certain features are observable only in one of the methods, they can be disregarded as artifacts, but if they appear in both methods, they can be accepted as ‘real’.

Q. Is that sufficient? A. Certainly not. The global consensus in the microscopy arena is that microscopists must be confident in their observations and their interpretations. In words of Betzig, a Nobel laureate in SRM: “resolution has to be such that the researchers have the ‘confidence of knowing’ that their view actually reaches down to the indicated level”. As one can see, the subjective perception of the observer plays a key role in the interpretation and meaning of the microscopic images. It is the inquiring mind of the microscopist what makes sense of the microscope’s representations. With that much subjectivity, how can we trust what scientists claim to observe through the microscope? The answer relies on an ethical code known as scientific research integrity.

Scientific research integrity and misconduct

There are infinite ways to undertake scientific research, and no universal method can rule all of them. To overcome the differences, scientists around the world share a series of values for the responsible conduct of research. According to the National Institutes of Health (NIH), research integrity includes:

• the use of honest and verifiable methods in proposing, performing, and evaluating research.
• reporting research results with particular attention to adherence to rules, regulations, guidelines,
• following commonly accepted professional codes or norms.

Any violation of the above code is defined as scientific misconduct. According to the US Office for Research Integrity (ORI), scientific research misconduct can be categorized into three groups:

• Fabrication: making up data or results and recording or reporting them.
• Falsification: manipulating research materials, equipment, or processes, or changing or omitting data or results such that the research is not accurately represented in the research record.
• Plagiarism: the appropriation of another person's ideas, processes, results, or words without giving appropriate credit.

In addition, the ORI clarifies that honest error or differences of opinion do not count as scientific misconduct.

Image integrity in modern microscopy

Traditionally, microscopists used to acquire micrographs by means of photographic films. Images were developed in a dark room through a series of chemical baths. When an image was not satisfactory for publication, the whole experiment was repeated and a new image was taken. Modern microscopy techniques, including SRM, employ cameras or photo-detectors to acquire the images. The information is spatially arranged in an XY matrix of individual elements (pixels) that are stored in a digital format with numerical values that represent the different intensities across the image. Digital images can be easily manipulated in multiple ways with the use of computational editing programs. As a result, modified images can alter the visual information submitted for scientific publication, constituting a case of research misconduct.

In recent years, the scientific community has expressed a growing concern about the lack of ground rules for appropriate digital image manipulation. In his 2010 article, Cromey summarizes a series of studies carried out between 2004 and 2008 in which prestigious journals reported inappropriate manipulation of at least one image in approximately 25% of accepted manuscripts. In addition, in a 2009 newsletter, the US governmental institution ORI, showed a steady increase of allegations involving image data, going from approximately 4% in 1993-1994, to 68% in 2007-2008. With such amount of cases, it is convenient to study the underlying reasons for this misconduct.

Reasons for inappropriate image data handling

Multiple reasons explain why researchers mishandle image data. In many cases, not a single cause but a combination of them precede the research integrity breach:

• Intentional reasons: though rarely seen in the science community, few researchers succumb to unethical practices such as image falsification and fabrication with the ambition of high-impact publications, founding or other personal interests.
• Ignorance: a large number of scientists are ignorant about the ethical guidelines of digital image manipulation (detailed in the next section). The lack of knowledge about what’s acceptable or not accounts for the most common reason for inappropriate image data handling among researchers.
• Sloppy experimentation: adjustments of the instrument during data acquisition (e. g. exposure time or illumination intensity) may lead to imaging artifacts that, by misinterpretation of researchers, are reported as real features.
• Beautification: scientists may perform manipulations of properly acquired data by means of easy-to-use image editing tools with the purpose of making a figure clearer, perfect and conforming to the expected result of the experiment.
• Pressure: in the urge of publication, scientists may beautify images to get away with figures that match their expectations, hence bypassing the ethical guidelines of image manipulation.
• Newcomers: the new generation of scientists are, in most cases, familiar with digital image editing tools such as Photoshop and regularly use functions for ‘cloning’ and ‘healing’, which are inadequate for handling scientific image data.
• Lack of supervision: with an increasing amount of workload, principal investigators have little or no time to check the quality of the images submitted by less experienced researchers for publication.
• Prejudices: many scientists believe that journal editors favor clean images with striking effects. This misconception leads to incorrect manipulation of the data to obtain the desired effect.
• Aesthetic perception: humans have a natural preference to observe well-organized, clean and nice-looking images.

Ethical guidelines for image integrity

Surprisingly, there are no unified rules for scientific image data handling. In their instructions to authors, the journals offer different guidelines for publication of figures. Some of them focus on the format rather than the content, where others also cover ethical considerations towards image integrity. Let us review some of them:

Nature: Images submitted with a manuscript for review should be minimally processed (for instance, to add arrows to a micrograph). Authors should retain their unprocessed data and metadata files, as editors may request them to aid in manuscript evaluation. A certain degree of image processing is acceptable for publication. The use of touch-up tools, such as cloning and healing tools in Photoshop, or any feature that deliberately obscures manipulations, is to be avoided. Processing (such as changing brightness and contrast) is appropriate only when it is applied equally across the entire image and is applied equally to controls. Contrast should not be adjusted so that data disappear. Excessive manipulations, such as processing to emphasize one region in the image at the expense of others (for example, through the use of a biased choice of threshold settings), is inappropriate. If "Pseudo-colouring" and nonlinear adjustment (for example "gamma changes") are used, this must be disclosed. Adjustments of individual colour channels are sometimes necessary on "merged" images, but this should be noted in the figure legend.

Journal of Cell Biology: Images should be minimally processed and accurately reflect the original data. We understand that image processing may be necessary and is appropriate in most instances. Our screening process examines the following: whether any specific feature within an image has been enhanced, obscured, moved, removed, or introduced; whether dividing lines are added between juxtaposed images taken from different parts of the same gel or from different gels, fields, or exposures; whether adjustments of brightness, contrast, or color balance have been applied to the entire image and that adjustments do not enhance, erase, or misrepresent any information present in the original, including the background. We also look for duplicated images within the manuscript; any reuse of images, including control data, across multiple figures should be explicitly stated and justified in the legend. Nonlinear adjustments (e. g. , changes to gamma settings) must be disclosed in the figure legend or Materials and methods section.

Journal of Molecular Cell Biology: Micrographs should be provided with a scale bar. All color images must be submitted in CMYK (Cyan-Magenta-Yellow-Black) color mode. The authors are responsible for ensuring that all figures correctly appear in black and white or in color.

Clearly, not having a well-defined set of rules difficult the researchers’ compliance to image integrity. Then, what other resources exist for the appropriate handling of digital image data? Despite all the concerns in the scientific community, a limited number of studies actually address this problem. The most comprehensive of them, Avoiding Twisted Pixels: Ethical Guidelines for the Appropriate Use and Manipulation of Scientific Digital Images, offers a detailed view of the problem and proposes a series of 12 guidelines for correct manipulation of images, with a thorough explanation of their underlying principles. SRM: the dilemma of choosing the right parametersIn order to surpass the instrument’s limitations, most SRM techniques depend on some form of computational reconstruction algorithm to analyze the raw image data and generate micrographs at the nanometer scale. A commonality between these algorithms is that the user – based on previous experience and knowledge – must select the appropriate parameters for correct image reconstruction. Moreover, the image reconstruction is sample-dependent, meaning that the same input parameters will render different results depending on the experimental conditions in which the raw images were taken. Here is the dilemma. How to choose the right reconstruction parameters without being bias by our subjective perception? Or, more importantly, where do we set the threshold between ethically acceptable and unacceptable? To my knowledge, there’s at present no active debate about this topic, yet it is fundamental to have it.

Conclusion

Subjective perception is part of human nature and, as such, will always be present in scientific observation. Over time, modern researchers have found mechanisms to filter out the observer’s bias and report the natural phenomena as objectively as possible. Yet, there is still work to be done. The introduction of new instruments and methods in the scientific arena requires a set of well-defined universally accepted guidelines so that all members of the scientific community play under the same rules. It is difficult to draw a line between ethically acceptable or unethical practice when it comes to digital image manipulation. A collaborative approach between physicists, biologists and mathematicians is, in this case, mandatory to interpret micrographs generated by computational algorithms, and to judge what conforms to reality. Yet, as Lambert and Walters suggest, “it will be likely never possible to have 100% certainty that SRM images are an accurate representation of the specimen”.

An active debate about image data manipulation must be encouraged among all stakeholders in academia. Researchers, teachers, students and journal editors shall share the existing ethical guidelines for image integrity and discuss the challenges to conform to them. This practice will ensure a critical thinking that will evolve into a more standardized way of scientific reporting.