Daily search and browsing activity logs of users, when self-annotated (in contrast to third-party annotations), can give us insights about the users’ information-seeking behavior.

The central idea of the paper is that important information (that is, ground truth) gets lost when log analysis annotation is done by third-party/external annotators (for example, which interactions belong to which task). The idea was to have the users annotate their search logs themselves.

The goals of this exploratory experiment-based research are stated as follows:

• “RQ1: From a user annotated search log with records of rich types of interactions, do we observe similar statistics of task sessions as those observed from an expert annotated ‘query-only’ log?”
• “RQ2: How do task characteristics relate to each other? And how do these characteristics co-occur within actual web user tasks?”

A browser extension was used to capture 23 users’ browser-based searching and browsing activities, for five days. The users annotated their own logs every day, and had the liberty to delete/exclude content they wished. Additionally, the users picked their top five to ten tasks and described those tasks’ 12 predefined characteristics (table 1), for example, frequency, complexity, length, collaboration level, and satisfaction.

The users’ task-based information-seeking behavior logs were broken down into three levels of granularity: complex task; subtask (logical session, that is, consecutive queries belonging to the same task); and physical session (all user queries or activities within a time window). Then, some descriptive statistics for each task were calculated and compared to related literature, for example, mean/median number of queries, timespan, and task boundaries.

Furthermore, the 12 user-described task characteristics were clustered and the authors applied correlation analysis, presented in table 4. The paper adds arbitrary interpretations of the found correlations.

The idea of studying user-annotated task-based information-seeking behavior in its natural habitat is arguably better than externally annotated studies. The related work section is presented concisely and with great attention to nuances in the three major related works that are discussed and referenced throughout the paper. The methodology section is described precisely in excruciating detail, and refers back to the three related works, which could be discussed in the discussion section rather than in the methodology section.

The major weaknesses of the paper are 1) over-dependency on previous work (because of the choice of RQ1) and constant comparisons; and 2) the unsubstantiated accuracy, rather than precision, of the findings and their interpretation. The correlations are interpreted in a causal sense, rather than as mere associations between two variables, which could be spurious. Additionally, each and every such causal interpretation can be interpreted in the reverse direction, using the exact same data correlation coefficients. It seems that postmortem explanations are appended and the interpretations are arbitrarily chosen based on intuition.