Haemodynamic Signatures of Temporal Integration of Visual Mirror Symmetry

Abstract

EEG, fMRI and TMS studies have implicated the extra-striate cortex, including the Lateral Occipital Cortex (LOC), in the processing of visual mirror symmetries. Recent research has found that the sustained posterior negativity (SPN), a symmetry specific electrophysiological response identified in the region of the LOC, is generated when temporally displaced asymmetric components are integrated into a symmetric whole. We aim to expand on this finding using dynamic dot-patterns with systematically increased intra-pair temporal delay to map the limits of temporal integration of visual mirror symmetry. To achieve this, we used functional near-infrared spectroscopy (fNIRS) which measures the changes in the haemodynamic response to stimulation using near infrared light. We show that a symmetry specific haemodynamic response can be identified following temporal integration of otherwise meaningless dot-patterns, and the magnitude of this response scales with the duration of temporal delay. These results contribute to our understanding of when and where mirror symmetry is processed in the visual system. Furthermore, we highlight fNIRS as a promising but so far underutilised method of studying the haemodynamics of mid-level visual processes in the brain.

1. Introduction

It is well established that visual mirror symmetry (hereafter symmetry) is a particularly attractive and salient stimulus with evolutionary importance [1]. Psychophysical studies consistently find that symmetric patterns are responded to faster and more accurately than asymmetric patterns [2] and observers frequently rate symmetric stimuli as meaningful, either as signifying an object or contributing to the aesthetics of an image [1,3,4]. This behavioural evidence for a general preference for symmetry is further supported by neuroimaging evidence of symmetry-specific neural responses from fMRI [5,6] and EEG [7]. These studies have consistently identified an increase in extra-striate cortical activation when presented with symmetric stimuli, including over the lateral occipital cortex (LOC) and areas such as V3 and V4 [5,6,8]. Neurostimulation methods have identified the LOC as causally involved in mirror symmetry perception, as TMS over this area significantly disrupts observers’ ability to differentiate symmetric and asymmetric dot patterns and faces [9,10]. The LOC is a functionally localised area involved in object perception, extending ventrally and dorsally along the lateral fusiform gyrus [11]. Activation in the LOC observed using fMRI has been found to scale with the salience of the symmetry in the image in a manner similar to changes in psychophysical performance [5,12,13]. For example, patterns with more symmetry axes [12,13] or a higher ratio of signal to noise elements [5] produce significantly greater BOLD response arising from neural activation. Furthermore, it has been consistently found that symmetry does not differentially stimulate early visual areas such as V1 or V2, and responses in these areas do not change with symmetry salience [6]. Together, these findings suggest that symmetry is preferentially processed by higher-level visual areas that are also involved in processing shapes and objects [11] and is not simply due to lower level processing of luminance or orientation.

Electrophysiological (EEG) studies have also identified a symmetry-specific response [7,14]. The Sustained Posterior Negativity (SPN) is a response occurring approximately 300 ms after stimulus onset [15,16]. Building on earlier fMRI and TMS studies, EEG source-localisation techniques have shown that the SPN is generated in the extra-striate cortex, generally in the vicinity of the LOC [17,18].

The SPN has been extensively investigated in relation to a range of stimulus features known to affect symmetry perception in behavioural studies. It is automatically generated by the presence of mirror symmetry in an array, even if symmetry detection is not the participant’s primary task and even when they are otherwise unaware of the presence of symmetry in the stimulus [16]. Like haemodynamic responses in fMRI, the magnitude of the SPN scales with symmetry salience, again in a manner consistent with behavioural studies [19]. Specifically, symmetric patterns with vertical axes generate a larger SPN than patterns with other axis orientations [20] and multi-axis patterns generate larger SPNs than single axis patterns [19,21]. Furthermore, the magnitude of the SPN is dependent on the proportion of symmetry signal in a given array; the higher the signal to noise ratio, the greater the amplitude of the SPN [19,21]. The SPN is also affected by other features such as luminance polarity [22,23]. Recently it has also been shown that the SPN is also produced in response to more complex but naturally occurring symmetric stimuli, such as flowers and landscapes [17].

In recent years, interest into the temporal aspects of visual mirror symmetry perception has grown in the psychophysical research community. Recent psychophysical studies have shown that symmetry can be recognised in otherwise random dot arrays as long as the stimulus onset asynchrony (SOA) between paired signal elements on each side of the symmetric axis is 60 ms or less [24]. If the temporal delay between onset of element one and the onset of element two in a pair is longer than 60 ms, observers can no longer discriminate between symmetric and random arrays [24,25]. Other studies have shown that symmetry information accrues with sequential presentations of novel symmetric information, whether this is individual symmetry pairs [26,27,28] or entire patterns [29]. Although the information being integrated varies somewhat across these experiments, they all replicate a similar 60 ms upper limit to the delay over which symmetry information can be integrated. Furthermore, in earlier work we have shown that symmetry detection scales with increasing temporal delay in a similar fashion to changes in proportion of signal to noise in static arrays [24]. As delay between element or pattern onsets increases, detection performance decreases (e.g., higher symmetry thresholds).

Given the consistent association between extra-striate cortical activation and symmetry salience, it is reasonable to suggest that a similar relationship exists between SPN magnitude and successful temporal integration [30,31]. Rampone et al. [30] were the first to show that the SPN is generated following temporal integration of otherwise asymmetric components. Using dynamically occluded symmetric polygons where no symmetry was present in the image at any point unless successfully integrated, they identified an SPN component approximately 300 ms after the second half of the shape was presented. When the occluded polygons were asymmetric, no SPN was identified, confirming that the response was symmetry-specific. Only one half of the polygon was visible to the viewer at any time, meaning that recognition of the symmetry of the shape was possible via temporal integration of the two halves. As the SPN was only generated after presentation of the second half of the stimulus when the two halves of the polygon were symmetric, this response therefore reflects successful temporal integration of symmetry information. If temporal integration did not occur, no symmetry is perceptible and thus no SPN is generated.

Generation of the SPN from temporally integrated symmetric patterns was replicated in a follow up study by the same authors [32]. Here, they again identified the SPN component 300 ms after onset of the second half of the stimulus in a dynamic-occlusion temporal integration paradigm. They also showed that the SPN can be compromised by changes in stimulus features, in particular use of a non-retinotopic reference frame and variation of the symmetry axis orientation, becoming smaller and shorter in duration. Rampone et al. [32] suggest this may be due to the additional computational processing demands imposed by these variations (e.g., changes in position). Furthermore, Rampone et al. [32] also showed that symmetry perception is a dynamic process that accrues information over time. Similar to psychophysical findings by Niimi et al. [29], Sharman and Gheorghiu [27,28] and Sharman et al. [26], Rampone et al. [32] showed that the SPN response is strengthened by sequential presentations of novel symmetry information. These two studies by Rampone et al. [30,32] are therefore the first electrophysiological evidence of temporal integration of visual mirror symmetry and show that the SPN is produced by successful integration but the magnitude of this response is dependent on the success and quality of the integrated information.

As discussed earlier, it has been shown that symmetry processing occurs in the extra-striate cortex, specifically in the vicinity of the LOC [5,6,10,33]. EEG source localisation techniques are consistent with this and show that the SPN is also generated in these regions [17]. However, there has been little additional investigation of the neural correlates of the temporal integration process. If it is found that temporal integration of mirror symmetry information occurs in regions of the extra-striate cortex such as the LOC, then this will provide important insights into the local and global processing of mirror symmetry, particularly in terms of temporal dynamics. Source localisation performed by Rampone and Makin [32] suggested that temporal integration of mirror symmetry was likely occurring in these extra-striate regions, which is consistent with previous research. However, the conclusions that can be drawn from such analyses are relatively limited given the low spatial resolution of EEG. One way to rectify this gap in the temporal integration literature would be to employ fMRI, which is generally regarded as the imaging gold-standard in terms of its spatial resolution [34,35]. fMRI has been employed in symmetry research previously [5,6,13,33], but its utility in this research space is somewhat limited due to high associated costs and low tolerability for some participants compared to other methods [36,37]. Further, compared to EEG techniques fMRI has a very low temporal resolution due to its relatively slow 3 s routine sampling rate [38]. These factors mean that fMRI is not well suited to studying the temporal features of a given process, particularly when it is occurring in the range of milliseconds like mirror symmetry integration.

Functional near-infrared spectroscopy (fNIRS) provides a potential third alternative to investigating neural processing of mirror symmetry. Like fMRI, fNIRS measures changes in oxygenated haemoglobin (HbO) in regions of the brain thought to be activated by a particular stimulus [39,40]. fNIRS works on the premise that light in the near-infrared spectrum is able to pass through biological tissue (including skin, bone, and cerebrospinal fluid) and enter the brain. Here the light is differentially absorbed by various chromophores, particularly oxygenated and deoxygenated haemoglobin (HbO and HbR). By calculating the difference between light intensity emitted and then absorbed at two different wavelengths, we are able to calculate changes in HbO and HbR in response to various stimuli [40,41]. Similar to the fMRI BOLD response, the fNIRS haemodynamic response function (HRF) reflects a relative change in HbO and HbR concentration in response to stimulation. HbR changes much less dramatically than HbO, but generally decreases over the same stimulation period due to a “washout” from the larger HbO increase [42]. The total change in haemoglobin relative to baseline levels (HbT) across the HRF can also be calculated, which accounts for both the change in HbO and HbR. This is most akin to the fMRI BOLD response and is generally considered the most stable marker of neuronal activity [40,41].

Although still a relatively new neuroimaging technique, fNIRS has been gaining popularity as a method of studying cognitive and perceptual processing [37,41,42]. Compared to fMRI, fNIRS is comparatively low-cost, flexible, and non-invasive. Furthermore, it provides a good balance of both spatial and temporal resolution; it has a higher spatial resolution but lower temporal resolution than EEG, and a lower spatial resolution but higher temporal resolution than fMRI [40,43]. While both fMRI and fNIRS rely on relatively slow haemodynamic response, fNIRS has a much higher routine sampling frequency [44]. Haemodynamic changes in response to neuronal activation generally occurs over 1–2 s. In fMRI, the BOLD response is sampled roughly every 3 s to capture the slow peak of the response window [38]. In fNIRS, sampling occurs every 0.01 s. Takahashi and Ogata [45] provided an early investigation into the application of fNIRS to study to visual system, specifically V1. They showed the classic HRF (increased HbO, with minimal negative change in HbR) in response to visual stimulation using a multi-channel fNIRS device. Following this, many different visual static and dynamic stimuli have been employed in fNIRS studies, including reversing checkerboards [46], moving patterns [47], faces [48] and patterns that varied in perceived depth [49]. Most of this research has focused on areas around V1; however Wijeakumar et al. [46] have used fNIRS to investigate areas outside of V1, including the extra-striate cortex around the regions of LOC.

Given the potential limitations associated with fMRI, as well as the comparatively low spatial resolution, fNIRS is a potential alternative to further investigation of the haemodynamics of mirror symmetry perception in the LOC, and in particular replicate and expand Rampone et al.’s [30,32] EEG studies of temporal integration of mirror symmetry. In this study, we report the first experiment, to our knowledge, where fNIRS has been applied to the study of mirror symmetry perception and temporal integration in the extra-striate cortex. We hypothesised that we would replicate the increased activation over the LOC in response to visual symmetry previously identified using EEG and fMRI. In the case of fNIRS, this would mean an increase in HbO relative to HbR in the symmetric condition when compared to the responses to noise stimuli. We also expected our findings to replicate and expand previous EEG [30,32] and psychophysical [24] studies of temporal integration of visual mirror symmetry. We use the same temporal integration paradigm used in Bellagarda et al. [24] which allows us to track how performance, or in this case magnitude of cortical response, varies with temporal delay duration. By using a dot pattern array where SOA is systematically varied within element pairs, we aim to on Rampone et al.’s [30,32] finding and explore the duration over which it is possible to integrate mirror symmetry and where in the cortex that integration might occur.

Based on these aims, we hypothesised that a symmetry specific haemodynamic response would be produced in the extra-striate cortex, specifically around the vicinity of the LOC, in response to symmetric stimuli, and the magnitude of this response would decline as temporal delay within element pairs increases. We thus hypothesised that the response would be largest for patterns where there is no delay. A smaller magnitude of response was hypothesised for patterns with a brief temporal delay for delays less than the upper integration limit of 60 ms [24,25,26,28]. When temporal delay exceeds this limit, no symmetry specific response was expected, similar to psychophysical findings where symmetric patterns with a long delay are perceptually indistinguishable from noise patterns with no delay [24,25].

2. Materials and Methods

2.1. Participants

Fifteen participants (nine female, mean age 30 years) from the University of Western Australia participated in this study. Fourteen of these participants were naïve to the purposes of the study. The research was approved by the Human Research Ethics Committee at the University of Western Australia, and informed consent was obtained from all participants. All had normal or corrected to normal visual acuity, confirmed using a LogMAR chart [50]. Participants also completed the Edinburgh Handedness Inventory [51]. Two participants were left-handed, but as control analyses indicated no significant effect of handedness, they were included in the final sample. One participant was not naïve to the purposes of the study; however their data was included in the final data set as no differences between their data and the wider sample could be identified. This participant is an experienced psychophysical observer and participated at the beginning of the data collection process.

2.2. fNIRS Set Up & Data Acquisition

fNIRS data were recorded using a Brainsight NIRS continuous-wave device. The Brainsight NIRS system uses laser diodes emitting 705 nm and 830 nm wavelengths. A custom montage of source-detector pairs was designed using fOLD software and the AAL2 atlas [52] to overlay V1 and extra-striate regions including the LOC. This was converted to the 10–20 EEG coordinate system used by the Brainsight software. Channels were positioned to cover the primary visual cortex and areas extending laterally and parietally. A total of 18 channels were used for each participant, composed of five sources and eight detectors, shown in Figure 1. In a 10–20 EEG coordinate system, extrastriate activity is detectable under scalp location PO7, PO8 and surrounds [3]. In comparison, V1 activity will see greater activation along the midline around Oz, O9 and O10. To determine whether this symmetry specific response was restricted to the occipital cortex, we also considered activation over the frontal regions. This montage contained one source and four detectors positioned over Fz (see Figure 1). This was to permit control analyses, particularly confirmation of no symmetry specific activity over non-occipital regions. The same cap montage was used for all participants. Detectors were positioned <30 mm from their respective sources.

Caps were positioned by centring the cap (determined to correspond to point Cz) relative to the midpoint of the distance from nasion to inion (i.e., centre of forehead to occiput on the back of the skull) and right to left tragus (i.e., left to right ear over the midline of the skull) for each participant. This is the same process commonly used for positioning 10–20 EEG caps. Individual head circumferences were approximately 54 cm. Once the cap was positioned and secured, hair was moved from underneath each source and detector to ensure optimum coupling and reduce impedance between the optode and the scalp. Acquisition parameters and signal quality were optimised between each run of trials. Detector sensitivity, gain and laser power were adjusted until best possible optode-scalp coupling was achieved and the heartbeat could be observed in the data traces.

2.3. Apparatus & Stimuli

Stimuli were generated using Matlab R2017B (Mathworks, Natick, MA, USA) and presented using Psychopy3 [53] via Cambridge Research Systems (CRS) BITS# (CRS, Kent, UK) visual stimulus generator displaying on a Sony Trinitron G529 monitor (screen resolution 1024 × 768 pixels, refresh rate 100 Hz). The stimuli used in this experiment were similar to those used in our previous psychophysical experiments on the same topic [24]. Each symmetric pattern was composed of 64 Gaussian dots positioned around a vertical axis. Example symmetric and random noise patterns are shown in Figure 2. Each dot had a diameter of 7′ of visual angle (half width at half height, standard deviation 6′). Dots had a luminance of either 0 cd/m² or 90 cd/m² on a grey scale background, and the mean luminance was 45 cd/m² with dots, therefore having a Weber contrast of 1 or −1, respectively. These dots were placed within a circular area with a radius of 6.4°. However, to prevent dots overlapping the edge of the screen, dot centres were restricted from occurring outside a radius of 6.13°. Furthermore, to avoid dots overlapping with each other, dots could not be placed within 30′ of visual angle of any other dot’s position or less than 15′ from the symmetry axis. Overall stimulus density was 0.49 dots per square degree but was slightly lower near the axis. Both symmetric and non-symmetric stimuli had the same distributions.

The circular field was divided into left- and right-halves. To form the symmetric stimuli, 32 dots were randomly placed on one side of the axis. The dots were then reflected over the vertical symmetry axis to ensure there were 64 dots arranged in 32 symmetrically positioned dot pairs. Both dots in a given pair always had the same luminance polarity. The asymmetric noise stimuli were generated in a similar way; 64 dots were randomly placed on one side the axis, and then reflected. One element from each pair was then removed such that there was no spatial pairing of the dots, but the total number of dots presented was maintained at 64 in total, consistent with the symmetric patterns. This design meant that noise dots could not occur in the mirrored position of any other dot in the array, controlling for accidental spatial pairings and eliminating spurious symmetry cues.

Stimuli were dynamic, and composed of 3000 unique frames, each of 10 ms duration. The total duration of each stimulus was therefore 30 s. Each dot was present on the screen for four frames (or 40 ms). On each frame, a quarter of the dot pairs were removed and replaced by new dots in a novel location. In stimuli that incorporate a delay, the placement of dots on the right side of the axis was delayed by a given number of frames relative to the left side. Reflection of the second symmetric dot therefore occurs a specified number of frames (delay duration ÷ 10) after the first dot in each pair are initially positioned on the left side of the symmetry axis. Delay durations were defined as SOA, meaning that the onset of element two in the pair was delayed by the specified amount of time relative to the onset of element one. Three delay durations were used: 0 ms, 50 ms, and 100 ms. The 0 ms delay condition is equivalent to a standard symmetry detection task but the stimuli remain dynamic. No temporal integration is required in this condition because all symmetry information is available simultaneously. The 100 ms delay was chosen based on evidence from psychophysical studies which consistently show that symmetry becomes undetectable when SOAs within element pairs exceed 60 ms [24,25]. The 50 ms delay condition, therefore, is near the upper threshold limit of delay tolerance for symmetry perception. Based on psychophysical and previous neuroimaging study findings, we would expect that there would be an attenuated haemodynamic response to the presence of symmetry in the 50 ms condition, while there should be little to no symmetry specific haemodynamic responses to the 100 ms conditions. The asymmetric stimuli were generated using the same procedure and had equivalent SOA durations between elements to maintain consistency of the temporal properties relative to the symmetric stimuli.

2.4. Experiment Design

One hundred unique 30 s video files were generated for the three temporal delay conditions. This was done for both the symmetric and asymmetric conditions, resulting in a bank of 600 dynamic stimuli. Each trial consisted of a 30 s uniform grey screen as a baseline period, followed by a 30 s stimulus. The 30 s presentation time for both the stimulus and baseline periods were chosen to allow sufficient time for the haemodynamic response to peak or return to baseline respectively. Each run of trials commenced with a longer 60 s baseline period to allow blood flow to return to resting state. Within a run, five symmetric and five asymmetric stimuli were presented, meaning that each run was approximately 10 min in duration. The symmetric and asymmetric stimuli were randomly interleaved such that no more than two of either stimulus type could be presented consecutively. Participants viewed two runs of each temporal delay condition. To avoid potential motor contamination, participants were not required to actively respond to the stimuli. Order of conditions was randomised for each participant. Total testing time was approximately one hour, not including set-up and breaks (approximately 20 to 30 min per participant).

2.5. Data Processing

Raw data for each run of trials (exported as a .nirs file) was processed using Homer2 [54]. Triggers were inserted manually into each data file to segment the data into symmetric, asymmetric and baseline periods. The first 200 s of each run was excluded as noisy data, including 140 s of baseline and approximately two 30 s stimulus presentations, plus an additional 20 s between the start of the fNIRS recording and the participants initiating the start of a run. A rapid normalisation of blood flow was observed across all participants that was reversed relative to changes in blood flow from 400 to 600 s. This generally occurs due to the need for the cerebrovascular system to “settle” into a resting state following a period of stimulation or exertion, and indicates substantial physiological artefacts are contaminating the signal of interest. The latter time period showed the most stable and reliable change in bloodflow, with no large shifts or obvious artefacts observed. As such, the data analysed below include data recorded from the latter 400 s of each run of trials to eliminate these global artefacts.

The raw intensity values were converted to optical density (OD). Individual channels with substantial amounts of observable noise were excluded from further processing; seven parieto-occipital channels and all four frontal channels were retained for each participant. Principle components analysis (PCA) filtering removed spatial eigenvector components accounting for 80% of the covariance across the data, which is associated with the presence of motion artefacts [55]. Approximately one component (or fewer) was removed from each data set. The data were then band-pass filtered to remove physiological artefacts. Chromophore concentrations were calculated using the Modified Beer Lambert Law (MBLL), with a partial pathlength factor (PPF) of 6.0. Changes in HbO and HbR were block averaged from two seconds before stimulus onset to 30 s after stimulus onset.

A group level haemodynamic response function (HRF) was extracted for each stimulus condition (symmetric, asymmetric and baseline) and chromophore type (HbO, HbR and HbT). The fNIRS sampling frequency of 10 Hz means that 300 data points were recorded for each channel over each 30 s stimulus window. Data from −2 to 0 s pre-stimulus onset were removed prior to statistical analyses as they do not contribute to the stimulus specific change in HRF. Difference scores were then calculated for each chromophore type by subtracting chromophore concentration during stimulus presentation from chromophore concentration at baseline for each sampling period. This processing pipeline is detailed in Figure 3 and was used for each delay duration individually.

3. Results

Our first analyses considered whether there was a significant change in the haemodynamic response over the total 30 s stimulus window for all delay durations. Typically, HbO will increase rapidly in the few seconds after stimulus onset and then plateau until stimulus offset. HbR instead tends to be relatively stable throughout the stimulus period. A set of Greenhouse–Geisser corrected two-way repeated measures ANOVAs with time and temporal delay duration as factors were consistent with this typical observation for each dependent variable (HbO, HbR and HbT). For HbO, there was a significant main effect of time (F(300, 600) = 1.61, p < 0.001, partial η² = 0.99) and a significant main effect of delay duration (F(1.02, 305.3) = 979.3, p < 0.001, partial η² = 0.771342). Considering HbR, there was a significant effect of delay duration (F(1.09, 326.1) = 1342, p < 0.001, partial η² = 0.89), but no significant effect of time from stimulus onset (F(300, 600) = 0.56, p = 0.99, partial η² = 0.2). HbT was similar to HbR, showing a significant main effect of delay duration (F(1.03, 310) = 1085, p < 0.001, partial η² = 0.82) but no effect of time (F(300, 600) = 1.15, p = 0.08 partial η² = 0.37). These results are consistent with expected changes in blood flow and chromophore concentration during sustained stimulus presentation. Importantly, we also show that this general response pattern does not change with increasing delay duration. Averaging the response across the total duration of the stimulus allows for this to be excluded as an additional variable in subsequent analyses substantially reducing the number of individual data points for analysis (30 s in 10 ms samples), while still preserving the relative differences in total response magnitude for HbO, HbT and HbR across delay durations.

One of our aims was to establish whether change in blood-flow was symmetry specific. Similar to findings with fMRI [5,6,33], we hypothesised that there would a greater change in HRF for the symmetric stimuli compared to the noise stimuli. Furthermore, based on the suggestion given by Makin et al. [21] that neural responses to symmetry scale with the perceptual goodness (or signal level of a given symmetric stimulus), we further hypothesised that the HRF’s magnitude will differ depending on the length of temporal delay in line with psychophysical findings [24]. If there is a greater response to the 0 ms and 50 ms delay conditions than the 100 ms delay condition, this would be evidence for temporal integration as the symmetry specific response can only be generated from the percept of global symmetry from integration of component elements. Three two-way repeated measures ANOVAs (one for each chromophore) comparing HRF magnitude for the symmetric and noise stimuli (2) and delay duration (3), showed significant main effects for both delay duration (HbO F(5, 12,600) = 1034, p < 0.001, partial η² = 0.29, HbR F(5, 12,600) = 714.7, p < 0.001, partial η² = 0.22, HbT F(5, 12,600) = 1057, p < 0.001, partial η² = 0.30) and channel location (HbO F(6, 12,600) = 2369, p < 0.001, partial η² = 0.53, HbR F(6, 12,600) = 519.5, p < 0.001, partial η² = 0.2, HbT F(6, 12,600) = 2603, p < 0.001, partial η² = 0.55). This analysis suggests that the magnitude of the response to symmetry is modulated by the duration of temporal delay in a given stimulus, as has been identified in prior psychophysical and EEG studies. Furthermore, the magnitude of the response varies across channel location, which may imply that the haemodynamic response is localised to a particular area. Follow up analyses are discussed below, in order to tease apart these main effects and better characterise how and where symmetry processing and temporal integration occurs.

Follow-up Tukey’s corrected t-tests were conducted on the total change in haemoglobin (HbT), which accounts for change in both HbO and HbR and is therefore most similar to the BOLD response investigated previously using fMRI. Results showed a significantly greater response to the symmetric stimuli compared to the noise stimuli when there was no temporal delay (t(12,600) = 6.55, p < 0.001, d = 0.05). There was also a significant difference between noise and symmetric stimuli in HbT in the 50 ms delay condition (t(12,600) = 9.4, p < 0.001, d = 0.06). Finally, as expected, there was no difference between noise and symmetric stimuli in HbT for the 100 ms delay condition (t(12,600) = 1.8, p = 0.8, d = 0.06). Previous psychophysical research has consistently found that the upper limit for successful temporal integration in mirror symmetry is a SOA of 60 ms. Our results replicate this consistent psychophysical finding in the HRF; when the limit of temporal integration is exceeded, symmetric stimuli become perceptually indistinguishable from noise stimuli, and therefore do not trigger the symmetry-specific increase in haemoglobin observed for patterns with no or very short SOAs.

Considering the identification of a symmetry specific response, we were also interested in whether this response was localised to the extra-striate cortex region previously suggested. Two-way repeated measures ANOVAs comparing magnitude of the haemodynamic responses to the symmetric stimuli for each chromophore indicated a significant main effect of delay duration (HbO F(12, 6300) = 1008, p < 0.001, partial η² = 0.66, HbR F(12, 6300) = 925, p < 0.001, partial η² = 0.64, HbT F(12, 6300) = 915, p < 0.001, partial η² = 0.64), as well as a significant main effect of channel location (HbO F(6, 6300) = 1095, p < 0.001, partial η² = 0.51, HbR F(6, 6300) = 719, p < 0.001, partial η² = 0.41, HbT F(6, 6300) = 1232, p < 0.001, 0.54). There was also a significant delay duration by channel location interaction (HbO F(2, 6300) = 1345, p < 0.001, partial η² = 0.30, HbR F(2, 6300) = 3294, p < 0.001, partial η² = 0.51, HbT F(2, 6300) = 1327, p < 0.001, partial η² = 0.30). fMRI and EEG studies have shown that symmetry is processed by the extra-striate cortex, specifically the right Lateral Occipital Cortex (LOC), and no symmetry specific response is observed in the V1/primary visual cortex [5,6,56]. As our channel locations extend across both V1 and the extra-striate cortex on both hemispheres, we would therefore expect to see differences in response magnitude depending on channel location.

Figure 4 and Figure 5 show that the magnitude of HBO and HbT response to symmetry varied significantly across channel locations where there was no temporal delay. There was a significant interaction between channel location and delay duration, as there was some localisation of response also evident in the 50 ms and 100 ms conditions, but it was comparatively much lower than the 0 ms delay condition. As would be expected given the established nature of HRF and BOLD responses, the HbR response was significantly smaller and less localised than HbO, especially in the 0 ms condition. Considering the specific channels, the greatest change in HbO is occurring over O2-PO7, O2-Pz and O1-PO4. This reflects a predominant left-hemisphere lateralisation of mirror symmetry processing and is consistent with symmetry specific activation being identified around PO7 in EEG studies [3]. Although this is initially surprising given that previous studies tend to show a right-hemisphere lateralisation, Rampone et al. [30,32] also show a specific increase in left-hemisphere activation in response to temporal integration of visual mirror symmetry. Importantly, in the 10–20 system these channels are positioned over regions of the extra-striate cortex and are generally consistent with the electrodes identified in previous EEG studies [3]. Grouped analyses were entered into AtlasViewer [57] and overlaid onto an MNI brain model. The variation in response magnitude for HbO and degree of localisation (Figure 5) was consistent with that observed in the statistical analyses, as expected. Response to noise patterns was significantly less localised, evidenced by the lack of significant areas of activation (dark blue regions in Figure 5) in the left-hand column.

Furthermore, regions of apparent activation that were present were found not to be significantly different from baseline in statistical analyses. As temporal delay between elements increased (top to bottom), HbO activation became significantly more generalised over the visual cortex and there is no statistically significant difference in activation across channels. This is consistent with a reduction in response specificity to symmetry coinciding with a reduction in the symmetry signal in the image caused the increase in temporal delay [21]. This supports psychophysical findings that as temporal delay increases, symmetry is still distinguishable to a certain point but with greater computational demands and lower sensitivity [24]. Importantly, our 100 ms noise, symmetry, and baseline responses in the bottom row are indistinguishable both in terms of the activation maps and statistical analyses. The presence of symmetry specific activation in the 50 ms delay condition, albeit reduced magnitude and more diffuse locations than the 0 ms delay, but not in the 100 ms delay condition is important haemodynamic support for the 60 ms upper limit to temporal integration of mirror symmetry. While our current data are limited in the specificity with which activation can be attributed to specific anatomical locations, the pattern of response and change in response observed across the visual areas of the cortex is consistent with a symmetry specific response that scales with length of temporal delay, as has been identified in previous psychophysical and EEG studies.

To compare responses to symmetry and noise stimuli in the frontal regions, three one-way ANOVAs were conducted (one for each chromophore). No significant differences were identified in any of the three chromophore types; HbO F(1.56, 4.67) = 0.22, p = 0.76, partial η² = 0.07, HbR F(1.33, 3.98) = 0.16, p = 0.77, partial η² = 0.05, HbT(1.71, 5.13) = 0.68, p = 0.52, partial η² = 0.18. Considering the responses to symmetric stimuli specifically, HRF did not differ from baseline (0 ms left panels on Figure 6 and Figure 7) and there is little variability across delay durations or chromophore types confirming the lack of symmetry specific haemodynamic response outside the occipital regions (e.g., in the frontal regions). Hemodynamic response functions for HbO over this region for each delay duration by chromophore type are illustrated in Figure 6 and Figure 7. Consideration of these figures shows no consistent variation in response magnitude relative to baseline for symmetric or noise patterns.

4. Discussion

While the presence of a symmetry specific electrophysiological response has been reliably reported [7,14,58], relatively little is known about this haemodynamic response to symmetry and how it relates to both EEG and psychophysical findings. The current study aimed to bridge this gap by investigating the utility of functional near infrared spectroscopy (fNIRS) for the study of visual mirror symmetry perception. In particular, we aimed to extend recent EEG findings by Rampone et al. [30,32] showing the symmetry-specific Sustained Posterior Negativity (SPN) is generated in regions of the extra-striate cortex following successful temporal integration of symmetric information. Consistent with our hypotheses, we identified an increase in oxygenated haemoglobin (HbO) over extra striate cortex in response to symmetric stimuli but not asymmetric noise stimuli. Further, we also found that the magnitude of this response scaled down with temporal delay, consistent with the 60 ms upper limit to temporal integration identified in psychophysical studies [24]. In sum, the results of this study make an independent contribution to our understanding of where and how mirror symmetry is processed in the visual system and demonstrate the utility of fNIRS as a promising alternative method of studying the haemodynamics of mid-level visual processes.

Together with Rampone et al. [30,32], this study is one of the first to show that temporal integration of otherwise meaningless information can produce a symmetry specific neural response. These studies provide further evidence in support of the flexible and dynamic nature of mirror symmetry perception already suggested by psychophysical studies [2]. Symmetry is often argued to have evolutionary significance as a way of explaining its perceptual salience [24,59,60,61]. If this is the case, then symmetry detection mechanisms need to be flexible enough to cope in dynamic, ever-changing real-world environments where in any given moment symmetric information may be partially occluded, in motion or otherwise imperfect. To cope with this variability, the system needs to be able to both effectively combine information across time, but also efficiently recognise core features of this aggregated information such as global mirror symmetry. Identification of symmetry-specific neural response in both EEG and fNIRS studies in the context of temporal delay provides support for this idea; the visual system is able to recognise meaningful structure in otherwise meaningless information via successful temporal integration. When temporal integration fails, such as when symmetry cues are separated in time by more than 60 ms, symmetry is not perceptible nor is a specific neural response generated.

Consistent with previous research, we have also shown that temporal integration of mirror symmetry is occurring in the extra-striate visual areas. Interestingly, while the symmetry specific response is quite localised, the estimated location is more medial than might be expected for the LOC [11,60]. EEG source localisation methods have also shown a somewhat more medial response to symmetry than fMRI studies [19,58]. The exact location of symmetry processing and involvement of the LOC warrants further investigation with more sophisticated fNIRS based localisation methods (e.g., DOT; see below), but our finding does support our initial hypothesis of a larger response to symmetric compared to noise patterns in the region of the extra-striate cortex. We also show that this activity is predominantly localised to the left hemisphere. While we are not the first to show a left-hemisphere localisation when perceiving dynamic symmetric patterns [30,32], most previous studies emphasise symmetry processing as occurring predominantly in the right hemisphere [9,10]. The picture is muddied further when considering fMRI studies often find bilateral involvement in symmetry perception [5,6,8]. Rampone et al. [30,32] argue that general symmetry processing is achieved by the LOC bilaterally, but a relative difference in activation of one hemisphere compared to the other may be observed depending on task demands. For example, in Bona et al.’s [9,10] studies, it may be that the strongly lateralised processing of faces in the right occipital face area (OFA) is thus recruiting the right LOC, but not left LOC processing, creating a relative difference in activation in one hemisphere. Rampone et al. [30,32] go on to argue that it is possible then that the additional demands associated with the requirement for temporal integration in our and Rampone et al.’s [30,32] tasks lead to the recruitment of more left hemisphere processing power and therefore relative localisation of the symmetry response to the left LOC in this context. Other features may also lead to variations in localisation of symmetry processing, such as symmetry in an object versus symmetry in an abstract pattern, or static symmetries versus symmetries in motion. Further imaging research is needed to explore this hypothesis and its explanatory power. FNIRS will also be a useful way of investigating variations in symmetry perception and temporal integration in different populations. This could include individuals with neurodevelopmental disorders (such as Autism Spectrum Disorder [62] or visual neglect following occipital lobe insult [63]). The results of this study underscore the uncertainties in our understanding of neural processing of mirror symmetry in visual cortex. More importantly, it highlights the need for flexible, non-invasive, and cost-effective methods of studying when, where and how symmetry is processed in the brain, a gap that may be bridged by fNIRS.

The current study provides novel evidence for the applicability of fNIRS for mirror symmetry and temporal integration experiments, and our results align closely with previous psychophysical, EEG and fMRI studies. This provides an important building block for future investigation into the anatomical correlates of mirror symmetry perception, and perception of higher-level stimuli more generally. Now that the suitability of fNIRS for investigating mirror symmetry has been established, it is possible to build on the results of the current study using more sophisticated experimental designs and analysis methods that would permit more detailed anatomical evidence of the location of activation to complement existing fMRI data. The use of diffuse optical tomography [64] and more complex anatomical registration techniques [65] would allow for increased precision in the localisation of responses and permit more sophisticated data processing and analyses techniques that were not possible within the context of the current experiment, such as short channel regression to remove physiological noise [66]. Such methods would permit a larger sample size than was achieved in the current experiment. These methods were restricted in the current study due to the available equipment and analysis software, which dictates the experimental design and possible analyses. Despite this, our study shows that fNIRS has the potential extend our current understanding of when, where, and how symmetry is processed in the brain.