Explicit coding in the brain: data-driven semantic analysis of human fMRI BOLD responses with Formal Concept Analysis

Dominik Endres, Ruth Adam, Uta Noppeney and Martin Giese

We investigated whether semantic information about object categories can be obtained from human fMRI BOLD responses with Formal Concept Analysis (FCA), an order-theoretic approach for the analysis of semantic information, such as specialization hierarchies and parts-based codes. Unlike other analysis methods (e.g. hierarchical clustering), FCA does not impose inappropriate structure on the data. FCA is a mathematical formulation of the explicit coding hypothesis [Foldiak, 2009, Curr.Biol., 19, R904-R906]. A human subject was scanned viewing 72 gray-scale pictures of animate and inanimate objects in a target detection task. To apply FCA, we employ a hierarchical Bayesian classifier, which maps fMRI responses onto binary attributes, and these onto object labels. The connectivity matrix between attributes and labels is the formal context for FCA. FCA revealed a clear dissociation between animate and inanimate objects in a high-level visual area (inferior temporal cortex, IT), with the inanimate category including plants. The inanimate category was subdivided into plants and non-plants when we increased the number of attributes extracted from the fMRI responses. FCA also displayed organizational differences between the IT and the primary visual cortex, V1. We show that familiarity and similarity ratings are strongly correlated with the attributes computed from the fMRI signal. Acknowledgements: this work was supported by BMBF FKZ: 01GQ1002, EC FP7-ICT grants TANGO 249858, AMARSi 248311, and DFG GI 305/4-1, DFG GZ: KA 1258/15-1 and the Max-Planck society.

Saccades bias mental arithmetic

Paola Binda, M. Concetta Morrone and Frank Bremmer

Evidence suggests that numerical quantities expressed symbolically and non-symbolically are processed by partially overlapping mechanisms. The numerosity of visual items is underestimated in the proximity of a saccadic eye movement (Binda et al., VisRes, 2011). Here we ask whether a similar distortion occurs for the processing of numeric symbols. Pairs of Arabic numerals (the operands) were flashed before or after a saccade; in a speeded task, subjects reported if the sum (or subtraction) was smaller than a probe digit presented 0.5s later. The percentage of ‘sum/subtraction smaller than probe’ responses was plotted against the difference between the probe and the true result of the operation, yielding psychometric curves. For operands presented just before saccades, curves were shifted left of 0, indicating that the result of the operation was systematically underestimated; judgments were unbiased for operands presented after saccades. This was observed for both sum and subtraction, and for both rightward and leftward saccades. We conclude that both symbolic and non-symbolic numerical quantities are underestimated in the proximity of saccades. A similar compression of visual space and time happens concurrently (Burr et al. TICS, 2010), suggesting that a common system processes magnitude along multiple dimensions, and may participate in visuo-motor coordination.

Experience in judging intent to harm modulates parahippocampal activity: An fMRI study with experienced CCTV operators

Karin Petrini, Phil McAleer, Catherine Neary, Julia Gillard and Frank E Pollick

Long experience with CCTV footage may lead to the formation of strong associations between complex visual stimuli and predicted behavioural outcome in social situations. To test this idea we scanned 15 CCTV operators and 15 age and gender matched novices while watching CCTV videos of 16 seconds and asked them to report whether each clip would end in violence or not. We carried out three separate whole brain analyses to examine differences in BOLD signal between the two groups based on 1) experimentally predefined clip types, 2) participants’ judgments during the scan, and 3) visual saliency as pre-assessed using eye-tracking. All three analyses consistently identified decreases in parahippocampal activity for expert CCTV operators when compared to novices. Whilst CCTV operators’ activity in the uncus increased with years of CCTV experience, activity in the posterior parahippocampal gyrus increased with scores of perspective-taking given by all participants. Based on the proximity of the uncus to the amygdala and the afferent projections the posterior parahippocampal gyrus receives from visual areas, we conclude that this region may work as a link between sensory and limbic information, a link that is not as necessary after strong associations between these kinds of information are formed.

In a self-paced shape perception task, subjects are underconfident, but make efficient use of temporal resources

Sven P. Heinrich and Michael Bach

How much time does a subject devote to each trial in a self-paced shape perception thresholding task, and how is this related to the subject's decision confidence? In a forced-choice task, subjects judged the orientation of Landolt C optotypes that were presented at different sizes at, above, and below threshold. The response times were recorded, and in each trial, subjects also rated how confident they were in their decision on orientation, using a four-point ordinal scale ("unconfident", "somewhat unconfident", "somewhat confident", and "confident"). Subjects performed on average 10% better than would be predicted by their confidence ratings. Response time was longest near threshold, and shorter above and below, with stimulus size, rather than confidence, being the primary determinant. This controverts the idea that response times are monotonously related to task difficulty, and instead supports Hogarth's law [Hogarth 1975, in: Utility, Probability, and Human Decision Making, D Wendt and C Vlek, Dordrecht, Reidel], which predicts response times to be relatively small for both easy and very difficult tasks. Such a behavior is efficient, because only around threshold the additional time spend on the task will result in better performance.

Dissociating developmental trajectories of part-specific and part-relational processing in object recognition

Martin Juttner, Dean Petters, Elley Wakui and Jules Davidoff

We have previously found evidence for a remarkably late consolidation of configural (part-relational) relative to part-based object recognition (Jüttner, Wakui, Petters, Kaur and Davidoff, 2012 Developmental Psychology, in press). Here we present a series of experiments that systematically confine the origin of the delayed development of part-relational processing. School children aged 7-12 and adults were tested in 3-AFC tasks to judge the correct appearance of upright and inverted presented newly learned multi-part objects that had been manipulated either in terms of individual parts or part relations. Manipulations were constrained to either categorical or metric changes of either individual parts or part relations. For categorical changes, even the youngest children were close to adult levels of performance for recognizing both changes of parts and part relations. By contrast, performance for recognizing metric changes of part relations was distinctly impaired in young children relative to that for recognizing metric changes of individual parts, and approached the latter not until 11-12 years. This distinctly retarded onset of part-relational processing was observed for both manipulations involving relative size and relative position. The results suggest a generic developmental dissociation of categorical and metric processing for part relations, but not for individual parts, in early adolescence.

Development of face-localization after extended congenital blindness

Tapan Gandhi, Piyush Swami, Amy Kalia, Garga Chatterjee and Pawan Sinha

The ability to localize faces in complex visual scenes is important for mediating many social interactions. Evidence that very young infants prefer to look at faces over non-faces has led to the conjecture that this ability might have innate roots. However, the operational difficulties involved in obtaining reliable responses from babies have limited rigorous experimentation to track the early stages of face localization skills. We have had a unique opportunity to address this issue through our work in Project Prakash with children who gain sight late in life. Here we report results from ten newly sighted children, ranging in age from 8 to 23 years. Subjects were presented with three versions of the face-localization task in complex natural scenes. The three conditions corresponded to 1. Faces shown with bodies, 2. Full heads and 3. Only the internal facial features in their normal configuration. We found that children showed poor face localization immediately after sight onset, favoring an empirical account of skill acquisition. Furthermore, longitudinal assessment of performance showed that bodies and external head contours were important cues over the early course of this developmental trajectory. These results demonstrate preserved plasticity for visual learning even late in life and, more specifically, also have implications for the mechanisms that subserve face-learning early in the developmental trajectory.