Towards Predicting a Realisation of an Information Need based on Brain Signals

The goal of Information Retrieval (IR) systems is to satisfy searchers' Information Need (IN). Our research focuses on next-generation IR engines, which can proactively detect, identify, and serve INs without receiving explicit queries. It is essential, therefore, to be able to detect when INs occur. Previous research has established that a realisation of INs physically manifests itself with specific brain activity. With this work we take the next step, showing that monitoring brain activity can lead to accurate predictions of a realisation of IN occurrence. We have conducted experiments whereby twenty-four participants performed a Q/A Task, while their brain activity was being monitored using functional Magnetic Resonance Imaging (fMRI) technology. The questions were selected and developed from the TREC-8 and TREC 2001 Q/A Tracks. We present two methods for predicting the realisation of an IN, i.e. Generalised method (GM) and Personalised method (PM). GM is based on the collective brain activity of all twenty-four participants in a predetermined set of brain regions known to be involved in representing a realisation of INs. PM is unique to each individual and employs a 'Searchlight' analysis to locate brain regions informative for distinguishing when a “specific” user realises an information need. The results of our study show that both methods were able to predict a realisation of an IN (statistically) significantly better than chance. Our results also show that PM (statistically) significantly outperformed GM in terms of prediction accuracy. These encouraging findings make the first fundamental step towards proactive IR engines based on brain signals.


INTRODUCTION
Information need (IN) is an essential concept and plays a core and fundamental role in the information seeking and retrieval process.Over the last several decades, much research has been dedicated to better understand IN to most effectively satisfy it with information retrieval engines, e.g.[8,12,23,32,58].In general, satisfying INs is a formidable challenge due to the inherent complexity and ambiguity associated with the IN concept.That is, expressing an IN using a set of query keywords is an uncertain and noisy process [53], as keywords can only vaguely approximate the actual IN [51].The problem becomes even more pronounced when an IN is ill-defined: i.e., when the searcher only knows "fringes of a gap in [his/her] knowledge" [12] making it extremely difficult for the searcher to identify and describe the IN precisely with keywords [6,8,11].Therefore, it is possible that a given query may not sufficiently define the characteristics of relevant documents, or even any relevant information since a searcher cannot form an appropriate initial state from which to form a query [13].
To close this gap, IR systems have been relying on techniques such as relevance feedback including explicit [29], implicit [57], and/or affective feedback [2] to better understand and satisfy users' information needs.Despite their invaluable contributions, these techniques lead to systems that are reactive in nature, being engaged in response to an input query issued by a user.Given the aforementioned problematic nature of users' queries (in terms of expressing searchers' real INs), recently, there is an increased interest in the research community in developing zero-query (or proactive) IR systems that aim to not only better understand, but even anticipate users' INs [9,49].These approaches mainly consider the contextual or behavioural information of users to predict their IN.While this is intuitively a promising approach, such proactive IR systems could potentially create an information overload situation for users, in particular, if users are not experiencing an IN when the information is presented to them.Therefore, being able to predict when an IN occurs is of high importance to better satisfying it.With this work, we make the first positive step in this direction.
Recently Moshfeghi et al. [42] have shown that there are clear, detectable, physical manifestations (i.e.neural correlates) of INs in human brains which can be identified in an early stage of an information seeking and retrieval process and also specific brain regions where INs manifest themselves.However, two fundamental research questions remain unanswered: • RQ1: "Is it possible to predict a realisation of an IN using brain signals?";• RQ2: "Can we rely on a common set of predefined brain regions to predict a realisation of an IN or is there a unique set of regions for every individual?"An answer to these questions will play a key role in opening new doors to the design and implementations of novel IR systems, which will be enabled to better and in particular proactively satisfies searchers' IN.To do so, we aim to train a classification model on brain activity to be able to discriminate between an IN realisation vs memory retrieval, within an IR process performed by participants engaged in a Question Answering (Q/A) retrieval task.In particular, in this paper, we are focusing on the period in which the brain exhibits complex activity with regards to the process analysing a question, from the moment of observing the question to the point in time when they either recognise an "Anomalous State of Knowledge" (ASK) [7,8], or they have a successful memory retrieval.We then aim (i) to build a model that predicts a realisation of IN (in its earliest possible state) from the signals of the brain regions that have been associated with an IN realisation and (ii) contrast its prediction accuracy with the accuracy of a prediction model that considers the signals of the whole brain regions, in a Q/A task.Our positive results can be the basis for next-generation proactive IR systems based on brain signals.
The remainder of the paper is organised as follows: Section 2 presents related work and the background.Section 3 discusses at length the experimental methodology.Section 4 presents and analyses our results.Finally, Section 5 presents our key conclusions.

RELATED WORK 2.1 From Reactive to Proactive IR
A well-known issue in IR research is that INs are inherently complex and ambiguous in that, typically, users may not know beforehand exactly what they are looking for, or, in other words, what are the set of documents/information that would satisfy their need [12].Given this, formulating queries (i.e., mapping INs to a set of keywords) is inherently an imperfect process [7,8,12,22,51].The formulated query is unavoidably therefore not guaranteed to be an accurate representation of the users' IN, and thus brings about the question whether the retrieved documents are relevant [55].
IR systems have addressed this issue using mechanisms based on relevance feedback.In short, relevance feedback is an iterative process whereby the system receives cues from users as to the relevance of presented results so far, improving the relevance of results presented in the next step in the process.Such cues from users come in many forms and types.As mentioned in Section 1, relevance feedback can be categorised as being explicit [29], implicit [57], and/or affective [2].The most robust method to improve retrieval effectiveness is explicit feedback [29].However, it places a large cognitive burden on users [56].Implicit feedback avoids this pitfall.In implicit feedback, relevance is indirectly inferred from the interactional data without users' additional input [24].Researchers, for example, have tried to infer relevance by understanding how task [57] dwell time [28] and click-through [24] relate to relevance.However, this is not a panacea.Kelly and Belkin [27] have shown that implicit feedback can be unreliable and difficult to interpret, limiting its usefulness.Affective feedback, on the other hand, [2] relies on different types of feedback data, such as facial expressions [5], eye tracking [36], and physiological signals [3,38] (such as skin temperature) and uses them as implicit relevance judgements.Nonetheless, affective feedback on its own is not yet considered to be very effective.More recently, Arapakis et al. used emotions in the process of information retrieval [2,4] and later showed that emotions can be used for building implicit feedback systems [4], and to personalise search [2].Going a step further, the work by Moshfeghi et al. used, in addition to emotions, [39], physiological features both for modelling relevance and for predicting task types [38].However, these approaches are reactive in nature, being engaged in response to an input query issued by a user.In this work, we aim to predict a realisation of IN before a user has issued a query.
Similar research efforts in the community have begun to explore and stimulate discussion about systems that can proactively anticipate and fulfil the information needs of users [10].In addition to the works published in [10], additional research in this domain of proactive IR is emerging.A recent example is the work by Song et al. [49] where a query-less search and recommendation engine was proposed.The engine predicts when and what types of tasks users will issue in the near future by considering the time dimension.They also considered a personalised approach for predictive IR by performing model adaptation by learning individual deep learning models for each user task.However, their proactive model is based on strong user behaviour repetition in search engine logs and even stronger repetition on mobile devices.An even more recent example is the work by Benetka et al. [9] which addresses the problem of identifying, ranking, and anticipating a user's information need, based on users' last activity.This work develops a method that gathers and ranks information needs relevant to an activity using a limited amount of query suggestions from a search engine.However, in this study, INs are inferred from users' check-in records on a location-based social network.
Our paper blends well with the above research efforts.By predicting a realisation of IN from brain signals, IR system could be in stronger position to disambiguate and understand INs, e.g. by taking into account contextual information/data around the time an IN was realised.It also could alleviate information overload scenario.That is, a proactive IR system could make a better decision as to provide any information to users, based on their prediction of a realisation of IN from their brain signals.

Advances in Neuroscience & IR
There is growing research literature using modern techniques of brain imaging to explore the user perspective of IR.The two topics which have received attention include relevance and information need (IN), though other research has begun to investigate the effect of task and specificity of search results [43].Research into the topic of relevance has been performed using the brain imaging techniques of Electroencephalography (EEG), magnetoencephalography (MEG) and fMRI.Results of Moshfeghi et al. [40] used fMRI to identify brain regions activated by the process of judging the relevance of an image.They contrasted brain activity from the viewing of relevant images to non-relevant images.Brain regions with higher activity when viewing relevant items included the inferior parietal lobe, inferior temporal gyrus and superior frontal gyrus.While these results were interpreted as representing the activity of visuospatial working memory on relevant items, the prefrontal cortex has also been shown to be activated in judging the relevance of written items [44] and thus this region could be involved in aspects of general task control and decision making.
Another study into relevance was conducted by Eugster et al. [17] that used EEG to show that the frequency content of the EEG signal as well as Event Related Potentials (ERPs) can be used effectively as a set of features to decode the relevance of a text.This work was later extended [16] to show that methods of automatic classification applied to single trials could be used to classify individual words being read as relevant or non-relevant successfully.Likewise, Kauppi et al. [25] showed using MEG that the frequency content of the MEG signal, along with eye movement data can be used for decoding the relevance of images.These results are consistent with the findings of Allegretti et al. [1] who reported on EEG results that indicated that within 500 ms EEG signals begin to appear that differentiate between viewing a relevant and a non-relevant image.These studies with fMRI, EEG and MEG have used the relative strength of the different measurement techniques to make great progress and to indicate where in the brain relevance judgments are happening and what the time course is of these neural processes that determine relevance.
Expanding upon work on relevance, research by Moshfeghi et al. [42] used fMRI to examine the neural processes involved in how IN emerges.In this study, they compared two tasks where participants read a question and were provided with responses that included the possibility of answering the question or asking for additional information (acknowledging an information need).In the first task, participants went on to the next trial regardless of their response, but in the second task they went on to an information retrieval dialogue if they stated an information need.Brain imaging results revealed the posterior cingulate as a region common to both tasks.However the posterior cingulate can be divided into different regions, and activity in different regions was found for the two tasks that reflect the role of the posterior cingulate in switching between internal (e.g.memory) and external (e.g.sensory) sources of information.
How the neural correlates of IN fit into the larger context of a search was explored by Moshfeghi and Pollick [41] who investigated search as a process of transitions between neural states.To achieve this, they defined the time periods corresponding to five different aspects of search (realisation of Information Need, Query Formulation, Query Submission, Relevance Judgment and Satisfaction Judgment) and the brain activity was contrasted between these different periods.Results showed how the cognitive demands of the different search periods involved various large-scale brain networks including perception, cognition and executive control that reflected task demands.For example, the transition between IN and Query Formulation revealed extensive changes consistent with what would be expected from the dynamic restructuring of neural resources to attempt to answer the query.Motivated by these results, this paper indicates that the period of IN involves complex patterns of brain activity and by application of more sensitive data analyses, reliable fine patterns of brain activity that are consistent with being in a state of IN can be revealed.

EXPERIMENTAL METHODOLOGY
To investigate our research questions mentioned in Section 1, an information need scenario was created where participants were asked a multiple-choice question, and if they did not know the answer, they engaged in a search process.Before describing the experimental methodology in detail, we would like to point out several practical constraints on the design and analysis of fMRI experiments that are critical to consider [15,40].Firstly, the fMRI scanning environment is restrictive in that a participant is placed in the central bore of the scanner and must lay supine with their head kept still.Moreover, only limited response/interactive devices can be placed in this scanning environment without causing signal or safety issues.This constraint led to the use of multiple choice questions for a task since it was possible to provide a response using an MRI-compatible button box.Another constraint is that although fMRI provides a spatial resolution of brain activity to within a few millimetres, it does not have a fine temporal resolution.This is because the time taken to achieve one measurement of the entire brain takes approximately 2 seconds.This relatively slow rate of data acquisition is compounded by the fact that the Blood Oxygenation Level Dependent (BOLD) signal measured is related to the underlying neural signal in a complex way that introduces further delays [19].As a consequence of these temporal factors, it was necessary to time the events happening in the experiment at a rate that was compatible with our fMRI measurements.To achieve a suitable design for our questions about IN we were guided by the methods used by Moshfeghi et al. [42] who explored the neural correlates of IN.These methods themselves had been adapted from research into problem-solving [59] that examined neural correlates of insight by comparing brain activity when a multiple choice response showed insight, to brain activity when a multiple choice response did not show insight.A final practical constraint is that running fMRI experiments are costly at £500 per hour which would limit the number of our participants.

Participants
Participants consisted of twenty-four healthy individuals with 11 males and 13 females.The participants were under the age of 44, with the largest group between the ages of 18-23 (54.1%) followed by a group between the ages of 30-35 (20.8%).The handedness survey indicated that 79.1% were right-handed, 12.5% were lefthanded, and 8.33% were mixed-handed.Participants tended to have a postgraduate degree (20.8%), bachelors (33.33%) or other qualifications (45.8%).They were primarily students (54.1%), though there were a number of individuals who were self-employed (20.8%), not employed (4.16%) or employed by a company or organisation (20.8%).Participants were primarily native speakers (79.1%) or had an advanced level of English (20.8%).They all had experience in searching, with an average of 11.66 years (SD of 3.58) experience.

Design
We used a within-subject design for this study.The independent variable was the realisation of information need, with two levels: an information need was realised (IN), and no information need was realised (no-IN), which was controlled by responding to questions viewed on the screen.The set of questions were designed so that averaged across all participants there would be an equal number of responses expressing an answer and expressing a need for search.The dependent variable was brain activity revealed by the BOLD signal.From this BOLD signal, we estimate the accuracy of different prediction methods of a realisation of an IN.

Task
The entire experimental procedure is explained in the following section, and here we explain the task performed by participants in an experimental trial while being scanned.Participants first were presented with a question for 4 seconds, then for 4 seconds, four possible responses were provided while the question remained on the screen.Participants were not able to make a response until after the 4 seconds of observing the possible responses.The reason for this was so that brain activity related to the motor response of pressing the button would not be contained in the target pattern of brain activity, which only included these first 8 seconds.After the 8 seconds, participants were able to respond.Of the four possible responses, one was always the correct answer, and one of them was always "need to search".If participants chose that they needed to search then, this trial was labelled as a "IN", and otherwise, the trial was labelled as "no-IN".The position of the four alternatives was randomised for each trial and the response provided by pressing one of the four buttons available on the button box that each participant had in their right hand.The time to respond was left free so that participants were not under time pressure to respond.The order of the questions was randomised for each participant.If the answer "need to search" was provided then they entered an additional stage where they formulated a search query (and submitted it verbally into a noise-cancelling microphone), received a document and evaluated this document.In the present study, we do not investigate brain activity during the search and document evaluation periods.Instead, we focus only on the brain activity in response to the presentation of the question and the possible responses.

Question Answering Dataset
To perform the task scenario mentioned in Section 3.3, we created a Question Answering dataset1 .To develop this standard set of questions, we used previous runs of TREC Q/A Track, in particular, we carefully selected a set of 80 questions from the TREC-8 and TREC-2001 Question Answering Tracks -Main Task2 .We chose these two Tracks since they were the first and last tracks where the questions presented there were (i) independent from one another, in contrast to other Tracks that share a relationship, and (ii) they also provided the correct answer to the questions.
We then manually examined all the questions presented in these two tracks and selected a subset of questions that (i) was not longer than one line, and (ii) the correct answer to the question was not longer than five words.This constraint is due to the limitation of presenting the questions and options to the participants in an fMRI setting.An additional constraint was that there were at least two relevant and non-relevant answers in their QRel.We then removed the questions that were ambiguous or were time dependent, e.g."Who is the president of Stanford University?" (TREC-8, Topic 51), making the answers provided in the Track not appropriate.The answers of all these questions were then checked by current search engines to make sure that the answers are still valid and correct.We also created two wrong answers for each question that were in the domain of the question.For example, for question "What is supernova?"(TREC-2001, Topic 1067) the correct answer is "An exploding star" and we created two other wrong answers i.e. "A newborn star" and "A dead star".We also made sure that the questions covered a wide range of topics, e.g.history, politics, science, etc. this was done to reduce any bias that might occur from an emphasis of a particular type of question.
Over this set of questions, two annotators separately judged the difficulty of the questions (i.e.hard or easy) and then selected a subset of questions where both annotators agreed upon their difficulties.For our task, 40 questions were randomly selected where 20 were easy, and 20 were hard questions.Since the fMRI data acquisition for these 40 questions was divided into two runs, additional care was made to further divide the questions across the runs so that they both had 10 easy and 10 hard questions covering a variety of topics.The goal of this procedure was to control the set of questions such that on average there was an equal chance of experiencing ASK [7,8] and knowing the answer.
Another extra step was to prepare the documents that were shown to the subjects once they engaged in a search process.This took the form of simulating a snippet answer that is returned by a current search engine such as Google when a question is submitted.For this purpose, we selected two relevant and two non-relevant documents from QRel.The length of the answers provided in TREC-8 and TREC 2001 were incompatible.To keep the size of the results consistent, for those answers that were too short, we found the original source file and selected sentences around the answer so that all snippets had the same length.The average length of the answers shown to the participants for the first and second run were 39.47 words (SD of 3.33) and 39.65 words (SD of 3.285) respectively.This was done to reduce any potential confounding effect of snippet size on the brain activity results.

Procedure
This section outlines the flow of the study, from beginning to end.Ethical permission for the study was obtained from the Ethics Committee of the College of Science and Engineering, University of Glasgow.Participants were recruited from the participant database at the Centre for Cognitive Neuroimaging, University of Glasgow.Participants were instructed that the experiment would take approximately one hour in total and would include performing tasks while being scanned, as well as a 10 minute period where the anatomical data were obtained, and they need only stay still.They were informed that they could leave at any point in time during the experiment and would still receive payment (the payment rate was £6/hr).They were then asked to sign a consent form.Before participating, participants underwent a safety check to guarantee that they did not possess any metal items inside or outside of their body, or any other contraindications for scanning, such as certain tattoo inks.They were then provided with clothes (similar to a training suit) to wear for the duration of the experiment as a precaution to avoid interference with the fMRI signal from any metal objects in their clothes.
Next, to provide training before entering the scanner, participants were given an example task and a corresponding set of example questions to familiarise themselves with the procedure.Once they had successfully completed their training task, participants entered the fMRI machine, and the experimenter adjusted the settings of the machine to maximise their comfort and vision.While being scanned, each participant participated in two separate runs of our task, with each run comprised of 20 questions.A short rest break given between the two runs was chosen to give the participants a break to relax during the scanning and to prevent fatigue.After the functional runs were complete, the anatomical data of each participant were obtained.
After completion of scanning, participants were asked to fill out an exit questionnaire.The purpose of this questionnaire was to obtain demographic and background information about the participants.This included familiarity with related fMRI and user studies as well as questions addressed at understanding the participants' subjective experience about how they found the experiment.They also filled out the Edinburgh handedness questionnaire [45], which provides the evaluation of whether the participant was right-, leftor mixed-handed.Handedness information was obtained since lateralisation of brain function is influenced by handedness, and we wished to ensure that our sample of participants approximated the general population.

Apparatus
Visual presentation of the questions, answers and search results to participants was obtained using Presentation ® software 3 to control projection on a LCD projector that projected onto a translucent screen, which was visible to participants in an angled mirror while they lay in the bore of the MRI scanner.

fMRI Data Acquisition and Preprocessing
A 3T Tim Trio Siemens MRI scanner, including a 32-channel head coil, was used to obtain all brain imaging data.This scanner is located on-site at the Centre for Cognitive Neuroimaging, University of Glasgow.Functional data included two T2*-weighted runs of our task with parameters: TR 2000ms; TE 30ms; 32 Slices; 3mm 3 voxel; FOV of 210, imaging matrix of 70 × 70.An anatomical scan was performed at the end of the scanning session that comprised a high-resolution T1-weighted anatomical scan using a 3D magnetisation prepared rapid acquisition gradient echo (ADNI-MPRAGE) T1-weighted sequence (192 slices; 1mm 3 voxel; Sagittal Slice; TR = 1900ms; TE = 2.52; 256 × 256 image resolution). 3Presentation ® software (Neurobehavioral systems, Inc.), http://www.neurobs.com.
The fMRI data were preprocessed using Brain Voyager QX.A standard pipeline of pre-processing of the data was performed for each participant [20].This involved slice scan time correction using trilinear interpolation based on information about the TR and the order of slice scanning.Three-dimensional motion correction was performed to detect and correct for small head movements by spatial alignment of all the volumes of a participant to their first volume by rigid body transformations.Also, linear trends in the data were removed, and high pass filtering with a cutoff of 0.0025 Hz performed to reduce artefact from low frequency physiological noise.The functional data were then coregistered with the anatomic data and spatially normalised into the common Talairach space [50].Finally, the functional data of each individual underwent spatial smoothing using a Gaussian kernel of 6mm.

Pilot Studies
Before commencing the main study, we performed a pilot study with two participants to verify the procedure worked appropriately.The results of the pilot study verified the accurate logging of both behavioural and brain imaging data and generally confirmed the effectiveness of our procedure.However, it also revealed some aspects that could be improved, and we changed the procedure accordingly.

Multi-voxel Pattern Analysis (MVPA)
The technique of Multi-voxel Pattern Analysis (MVPA) is increasingly used to analyse fMRI data, aiming to identify patterns from brain activity and enabling sensitive identification of the information encoded by brain activity.Most early fMRI studies investigating perception and cognition employed a General Linear Model (GLM) and aimed to localise brain regions by employing a univariate analysis at every voxel to contrast the magnitude of the brain response to different stimuli conditions.In contrast, MVPA employs a multivariate analysis that can examine the spatial encoding of information in a set of voxels.Increased performance over the univariate GLM approach is obtained since brain activity related to a mental process can be distributed across multiple brain areas [33], and MVPA is sensitive to these differences.

IN Prediction Methods using MVPA
For prediction of IN realisation, we have a binomial classification problem where the classes are "+1" (IN) and "-1" (no-IN).Therefore, we used ROI 4 -SVM, an implementation of SVM for MVPA in Brain Voyager QX 5 to discriminate between the two classes explained above.We trained our models using a linear kernel which outperformed other SVM kernels (e.g.polynomial and RBF) based on our analysis, not presented due to the space limits. 6Using the settings described above, we devised two prediction methods, a Generalised and a Personalised.

Generalised Method (GM):
The Generalised method follows the hypothesis that there is a common ROI for all participants that represents an IN realisation.We created this ROI by including spheres of radius 5mm centred around each of the Talairach coordinates of peak activation reported by [42] in their GLM analyses contrasting IN versus no-IN.All reported regions were selected regardless of whether they showed greater activity for IN or no-IN for comprehensiveness.
Personalised Method (PM): In contrast, the Personalised method follows the hypothesis that there is a unique ROI for each individual that represents an IN realisation as revealed by the Searchlight analysis [30,31].The Searchlight analysis uses MVPA and addresses the issue that the entire collection of brain voxels forms a high dimensional space.To reduce dimensionality and establish a reduced set of feature voxels it works by moving a small predefined volume (three voxels in our case) throughout the entire brain and creates a map based on the prediction accuracy obtained at the centre of the volume.From this map, it is possible to find all voxels with performance above a criterion value to define a personalised ROI that can then be used to test the prediction accuracy.
Baseline (BL): Our baseline represents an untrained model where all its predictions are based on a random choice, i.e.where its accuracy is set to 50%.

Task Perception
In this section, we investigate participants' evaluation of the task they had performed in terms of the judged difficulty of the task, the familiarity of the participant with the task and the degree to which they found their performance of the task to be stressful, clear, successful, and satisfactory.Namely, participants were given the following questions "The tasks we asked you to perform were [easy/stressful/familiar/clear/Satisfactory] (answer: 1: "Strongly Disagree", 2: "Disagree", 3: "Neutral", 4: "Agree", 5: "Strongly Agree")".Descriptive statistics of these responses are shown by box plots in Figure 1, which show five key statistics: the minimum, first, second (median), third, and maximum quartiles. 7.These results indicate that participants found the tasks difficult (not easy) and stressful, familiar, clear, successful, and satisfactory.

Log Analysis
The fMRI analysis relied upon a participant's response to the question to code whether a trial was IN or No-IN.This raised the consideration that for analysis of the brain data to have the maximum

Comparison of Brain Regions Associated with GM and PM
Before examining the performance of the MVPA approach in classification accuracy of discriminating between IN and no-IN states, it is instructive to examine the brain areas used by the two approaches.The GM technique restricted the analysis to the same feature set of voxels for all participants while the PM used a unique feature set of voxels for each individual as revealed by the Searchlight procedure.Given the bespoke nature of PM, where a unique feature set of voxels was derived for each participant, comparison of the regions used in PM to those used in GM is not straightforward.To address this issue, we created a probability map of the individual PM results that allowed us to visualise how many of the twenty-four participants had the same voxel present from the Searchlight procedure.To achieve this, we first defined 70% as a common threshold of prediction performance for the Searchlight procedure in discriminating IN versus no-IN.This probability map included 87.9% of all voxels in the brain volume if we examined the probability that a voxel would be found in any one of the participants.Since this map contains almost all voxels in the brain volume, we aimed to identify a more specific subset of voxels by increasing the criterion in the probability map to 0.35, corresponding approximately to a voxel being found in 8 of the 24 participants.This resulted in a map with 0.6% of all voxels in the brain volume.This map is shown in Figure 2, along with the regions used for the GM analysis.Details of the voxels identified in our probability map are provided in Table 1.As can be seen, the probability map revealed ten areas that are distributed across the brain, in many cases bilaterally, and concentrated mostly in frontal and occipital regions.
Additionally, except for the inferior frontal gyrus, there is little overlap between the GM regions and the PM regions.An essential point here is that the current results used a Multi-Voxel Pattern Analysis (MVPA) while the previous experiment used a General Linear Model (GLM) analysis.MVPA uses the pattern of activation of a group of voxels to classify brain activity and is sensitive to the spatial pattern of information encoded in brain activity.However, GLM analysis focuses on brain activity that occurs per each   voxel and is sensitive to differences in mean activation [14,21,52].Therefore, MVPA and GLM analysis are different in nature, and it is not entirely expected that they would yield identical results.Since MVPA considers multiple voxels at the same time, it can lead to the identification of additional regions that collectively have a discriminative power but not individually (which is the case for GLM).Thus, the areas reported by Moshfeghi et al. [42] but not found in our PM results would be those with significant differences in mean activation and not necessarily where the spatial pattern of activation could be decoded.However, one region was highlighted in both studies, i.e. the left inferior frontal gyrus region, which indicates that this region both changes its mean activation and its spatial distribution of activity with IN.Future research can examine this property more closely.With this in mind, we now describe the voxels presented in Table 1.The ten areas presented in Table 1 are commonly related to the domains of text/semantic processing (precentral gyrus, right inferior frontal gyrus), vision (middle occipital gyrus, lingual gyrus, fusiform gyrus), complex cognition (medial frontal gyrus), executive function (left inferior frontal gyrus) and reward (middle frontal gyrus).The finding of areas involved in complex cognition suggests that a realisation of an IN involves high-level cognition [23].For example, the medial frontal cortex (Brodmann Area 10) is at the most anterior section of the frontal cortex.Models of branching control of cognition suggest this area serves at the top of a processing hierarchy to assemble information across different processing strands.Similarly, the executive function includes processes such as working memory and keeping track of task rules.Thus the region of left inferior frontal gyrus, known as the dorsolateral prefrontal cortex, suggests that there is a difference in executive function depending upon whether there is an IN or no-IN.However, an unexpected finding is that a realisation of an IN need appears to be encoded in visual and textual processing areas.While these areas are consistent with performing the required tasks of reading questions and answers, It is not clear why it should encode a realisation of an IN.We would like to investigate this in the future.In summary, it would seem that these ten regions are distributed across the brain and involve both (i) high-level cognition, which could represent the concept of IN per se, and (ii) low level perceptual and textual processing where the decoding results from differences in the depth of processing IN vs no-IN items.

Prediction Accuracy of IN Realisation
Table 2 shows the classification performance averaged over the twenty-four participants of the study for both the GM condition when a common feature set of voxels was used across all participants and for the PM condition when a unique feature set of voxels was obtained for each participant.The table reports the accuracy of the model (i.e. the fraction of items in the test set for which the models' predictions were correct) using 10-fold cross-validation.We also performed a paired Wilcoxon test between the predictions obtained for each model to check the significance of the difference with its baseline.We use (**) and ( † †) to denote the fact that a model trained on a set of features had results different from that of "BL" and "GM" with the confidence level (p < 0.01) respectively.The key findings arising from our study are that brain activity could be used to predict whether participants were realising an IN (addressing RQ1).If the brain activity of a set of common regions to all participants was used (i.e.GM) then performance was (statistically) significantly above chance.In addition, the prediction accuracy (statistically) significantly improved over both chance and GM when the brain activity of a set of unique regions to each participant was used (i.e.PM) (addressing RQ2).Our findings are encouraging and indicate that it is possible to build proactive IR systems in which the IN realisation can be detected from brain signals.

DISCUSSION AND CONCLUSION
The main goal of Information Retrieval (IR) systems is to satisfy searchers' Information Needs (IN).However, satisfying INs is a formidable challenge, due to the inherent complexity and ambiguity associated with the IN concept.As a result of this, searchers' queries (i.e.searchers' representation of their IN) may not sufficiently define the characteristics of relevant documents or even any relevant information [13].Recently, there is an increased interest in the research community in developing zero-query (or proactive) IR systems aiming to not only better understand, but even anticipate users' INs [9,49].However, current approaches mainly consider the contextual or behavioural information of users to predict their IN.While this is intuitively a promising approach, it could potentially create an information overload situation for users, in particular, if users are not experiencing an IN when the information is presented to them.It is essential, therefore, to be able to detect when INs occur.
In this paper, we investigated the possibility of predicting a realisation of an IN from brain signals.Previous research has established that INs physically manifest themselves with specific brain activity.With this work we take the next step, showing that monitoring brain activity can lead to accurate predictions of IN occurrence.We have conducted a "within-subjects" design experiment where we measured the brain activity of twenty-four participants while they performed a Question Answering (Q/A) Task.The independent variable was the realisation of an information need, with two levels: an information need was realised (IN), and no information need was realised (no-IN), and the dependent variable was the BOLD signal.From the TREC-8 and TREC 2001 Q/A Tracks, a set of questions was carefully designed so that we obtained an equal number of responses for expressing an answer to the question and expressing a need to search for more information to answer the question.We then test two methods for predicting a realisation of an information need.A Generalised method (GM) that uses a predefined set of brain regions for all participants as well as a Personalised method (PM) that uses a unique set of brain regions for each participant.
Given that PM provided a unique ROI for each individual, we then investigated the probability that the ROIs obtained for each individual were overlapping, presented as a probability map.When we used a criterion that a voxel can enter the map if at least one participant obtained this voxel, 87.9% of all voxels in the brain volume were included in the map.This is consistent with previous studies suggesting that representational and memory systems are widely distributed in the brain [48].When we increased the criterion to 0.35 to examine highly overlapping regions, 0.6% of all voxels in the brain volume were included in the map, revealing ten regions.An analysis of these ten regions suggested that these regions are distributed across the brain and involve both (i) high-level cognition, which could represent the concept of IN per se, and (ii) low level perceptual and textual processing where the decoding results from differences in the depth of processing IN vs no-IN items.Such a finding is consistent with theoretical views that information processing in the brain involves extensive feedforward and feedback between brain regions [18].
Our findings show that brain activity could be used to predict whether participants were in a state of information need (addressing RQ1).The prediction accuracy depended upon the choice of brain regions used to learn the model.If the brain activity of a set of common regions to all participants was used (i.e.GM) then performance was (statistically) significantly above chance.Also, the prediction accuracy (statistically) significantly improved over both chance and GM when the brain activity of a set of unique regions to each participant was used (i.e.PM) (addressing RQ2).This result shows the benefit of PM for the prediction of a realisation of IN from brain activity.
The model proposed by Moshfeghi et al. [42] postulated that activation in the dorsal posterior cingulate cortex (PCC) represented a switch to broad attention directed to the external world [35] and this was an important neural correlate of a realisation of an IN.The current results advance our understanding of this model.Firstly, functional connectivity exists between the PCC to the inferior frontal gyrus (dorsolateral prefrontal cortex) [34], and anatomical connectivity exists between the PCC and the medial frontal cortex [46,47].Thus, the encoding of a realisation of an IN in these highly cognitive regions can be regulated by activity in the PCC.In addition, the current finding that visual and textual regions encode a realisation of an IN are consistent with the modulatory activity of the PCC in exerting top-down influences on perception through the regulation of extended brain networks.
In conclusion, the results of our experiment not only show that we can predict a realisation of an IN from brain activity but also substantially advance our understanding of the neural mechanisms involved in such a phenomenon.The encouraging results obtained from GM and especially PM in predicting a realisation of an IN provides an essential and fundamental step towards the next-generation of proactive IR systems based on brain signals.These results point to the beneficial use of fMRI to examine activity within the entire brain with high spatial resolution to reveal areas involved in a realisation of an IN.Given that this paper is the first attempt to predict a realisation of an IN from brain signals, such a high spatial resolution was crucial to investigate our hypotheses thoroughly.
Our findings also inform what other neuroimaging techniques might be effective for further study of a realisation of an IN.This is because brain imaging technologies have varied effectiveness in revealing activity at different depth levels in the brain.For example, the left dorsolateral prefrontal cortex, revealed in the present study, is a brain region that is relatively superficial and thus amenable to study with a variety of other brain imaging technologies.In future work, we want to continue to advance our understanding of brain activity for a realisation of an IN across other IR scenarios.Further, we would like to translate our current findings to mobile brain imaging technologies, such as EEG or functional Near-Infrared Spectroscopy (fNIRS) and possibly build proactive search engines that use brain activity for realising the occurrence of an IN.

Figure 1 :
Figure 1: Box plot of the task perception based on the information gathered from the questionnaires of twenty-four participants.The red diamond represents the mean value.

Figure 2 :
Figure 2: Comparison of brain regions in PM with probability greater than 0.35 to results from GM based on regions found in [42] PM and GM regions are shown in orange and green respectively.

Table 1 :
Details of probability map of PM results, including their anatomic label, location, Brodmann Area (BA), effect size and volume.

Table 2 :
This table shows the prediction accuracy of our methods (presented as columns).The percentage of improvement over the baseline is presented in the parenthesis.The best performing model is highlighted in bold.