Making the News: Digital Creativity Support for Journalists

Abstract

This paper reports the design and first evaluations of new digital support for journalists to discover and examine creative angles on news stories under development. The support integrated creative news search algorithms, interactive creative sparks and reusable concept cards into one daily work tool of journalists. The first evaluations of INJECT by journalists in their places of work to write published news stories revealed that the journalists generated new angles on existing stories rather than new stories, changed their writing behaviour, and reported evidence that INJECT use had the potential to increase the objectivity and the boldness of journalism methods used.

Introduction

Journalism involves the search for [3] and critical analysis of information [21] to create news stories and reports. How journalists discover and select sources of this information is important, both to avoid bias and to be credible and trusted. However, discovering and examining information sources takes time – time that journalists increasingly lack as news and media organizations are squeezed by reducing circulations, revenues and staff numbers [29]. As a consequence, many journalists use subsets of available and familiar information sources to create stories in the limited time available. However, this behaviour that can reduce the number and diversity of angles used to report news, as well as weaken the role the journalism in democracies [3] because citizens lack enough information to hold the powerful to account.

Journalism is one of the creative industries. Most journalists exhibit professional-level creativity because their news stories generate income and provide a living [14]. However, support for journalist creative thinking is limited. Most content management systems and search engines only have keyword search functions to discover information sources [18], and require journalists already to know and describe the creative angles to investigate. Interactive creativity support tools exist for different creative industries (e.g. [1, 13)], but newsrooms lack the digital support to discover possible creative angles on news stories, i.e. stories that are novel and have value to their organisations, their readers and their democracies [3].

This paper reports new digital creativity support for use by journalists. The tool, called INJECT, integrated natural language processing, creativity search algorithms and interactive creative sparks. To support use by journalists, INJECT was implemented as an Add-on sidebar for the Google Docs text editor often used by journalists, so that its creativity support was visible next to the story being written. The support had the potential to both relieve journalists of a demanding process that is difficult to do well under time pressure, as well as to expand possibilities open to journalists. The next 3 sections summarize the creativity challenges that newsrooms face, report digital creativity support generated in different creative industries, and define creativity in newsroom journalism. The paper then describes INJECT’s interactive creativity support and software architecture, and how the creativity support was designed within the Add-on sidebar’s constraints. It then reports results from first evaluations of the effect of INJECT use on how journalists discovered news stories. It ends with threats to the validity of the early evaluation results, limitations and conclusions, and how INJECT might be deployed more effectively in newsrooms.

Related work

Surprisingly few studies to inform the design of new digital support for journalists have been reported. Exceptions were the design implications for future tools to discover local news information sources reported in [9], and the Maater system that corrected news misinformation using high-ranking crowd-sourced entries [16]. To workaround the lack of digital tools to support creativity, some journalists have adopted general-purpose ones such as import.io and www.social-searcher.com that keyword-search multiple social media channels but not news information sources, and present comparative results. However, these tools were neither tailored to support journalist tasks nor provided explicit support for discovering creative angles on stories. Other digital journalism tools have implemented artificial intelligence techniques. For example, the Alchemy API was developed to support journalists to make sense of unstructured natural language data and generate human insights using text analysis and visualization mechanisms. Likewise the NewsReader tool implemented text analysis and artificial intelligence mechanisms to build structured event indexes of large volumes of financial and economic data for decision making from news content [23]. However, none of these tools had capabilities to search information sources to discover possible creative angles on future stories, or to support human creative thinking to examine these angles.

Unlike in journalism, digital creativity support has been implemented for professionals in other creative industries such as the performing arts, music, and film and television [1, 13]. Examples of the digital support include StoryCrate, a collaborative editing tool developed to drive users’ creative workflows within a location-based television production environment [4] and Trigger Shift, which appropriated information technologies into performance art in theatre [25]. More recent studies have revealed the need to integrate this digital creativity support into daily work practices. Not only does this avoid imposing additional cognitive load on professionals, but it also supports pain-free experimentation [10]. It can also deliver simultaneous productivity and creativity benefits to these professionals [17].

Today’s crisis in journalism

The digitalization of the production, distribution and consumption of news has led many news businesses to become uncompetitive [6]. The result has been a decade-long crisis in journalism. Even though this crisis has required news businesses to operate more competitively [24], many work practices have remained unchanged, due in part to the conservative attitudes of journalists [7]. Newsrooms have not adopted many new digital tools, even though the need to discover and examine information from multiple sources had been recognized widely (e.g. [30]).
That said, case studies reveal that journalists are still motivated by opportunities to develop new skills, including new creative skills [19]. Indeed, investigative and visual journalism demand new forms of creative search and association. Computational exploration in journalism increasingly involves creative thinking at the intersection of journalism and data technology. This creative thinking transcends geographical, disciplinary and linguistic boundaries [11]. Multi-skilled journalists have more control over their work and are more creative [33], although the high demands of daily production mean that many have struggled to develop these creative capabilities to their full potential.

Amabile identified that people who are not expert in creativity work need task motivation, domain-specific skills and creativity skills to produce creative outcomes [2]. Whilst most journalists are motivated and have journalism skills, most journalists also lack the creativity skills that are needed [19]. Therefore, inspired and informed by the challenge to design new digital tools for journalists that exploit opportunities to support their development and use of creative thinking skills, a new interdisciplinary collaboration between journalists and computer scientists formed.

The new collaboration developed digital creativity support for newsroom journalists to: R1) discover angles on news stories that are more creative than at present, from more information sources, in order to be more credible and trusted; R2) discover new angles more productively than at present, to be creative with less resources in order to enable their newsrooms to operate more competitively [24]; R3) use the creativity support in their everyday work tools, to encourage more journalists to adopt it, and; R4) learn good creative practices of more experienced journalists, because journalists seek new creative thinking skills [19]. The focus was on support for rather than automation of the work of journalists – journalists using the digital support were still expected to browse news information, discover angles on stories and write stories with these angles. This paper reports the research to achieve requirements R1, R2 and R3, and first evaluations of the new digital creativity support by journalists to investigate 2 research questions.

Reframing journalism as little-c creativity

To design the digital support needed by journalists, the collaboration developed a definition of creativity that was specialized to newsroom journalism. Although many journalists engage in professional-level creativity [14], much of their daily work writing news stories can be characterized as little-c creativity – daily activities not perceived to be creative [14], and undertaken by people who are not expert with creativity skills. Therefore, building on established definitions that creativity produces outcomes that are novel and appropriate to the task [27], little-c creativity in newsroom journalism was defined as activities to write news stories that are novel to the journalist and appropriate for publication. Furthermore, to distinguish these activities from straightforward news reporting, the journalist uses his or her creativity skills to generate the knowledge incorporated into the news stories and/or use in the activities to write the news stories, to render the stories novel. This contrasts with traditional news reporting, in which the journalist acts primarily as a stenographer who reports facts as accurately as possible, and does not introduce new knowledge that s/he generates.
For example, a journalist on a local newspaper who reports important facts from a town hall meeting that might be new to the journalist, such as a planning decision, is not engaged in little-c creativity because no creativity skills are used and no new knowledge is generated. In contrast, if the journalist uses creativity skills to generate new knowledge about, for example, a connection between this decision and the business interests of a politician, and this new angle is appropriate for publication, then the journalist exhibits little-c creativity in newsroom journalism. Likewise, if the journalist interviews the politician to acquire more information based on the knowledge generated, then s/he is also exhibiting little-c creativity in newsroom journalism.

However, as newsrooms lack the resources [6] to train journalists to acquire creativity skills using traditional methods, a new digital tool was investigated to support newsroom journalists to use simple creativity skills interactively to discover creative angles on news stories more productively using their everyday work tools.

INJECT’s digital crativity support

INJECT is a new digital support tool for individual journalists to discover and examine creative angles on news stories under development. It was built to support human-centred creative cognition [15], in which idea generation about new angles took place concurrently with information search. INJECT searched news information with creative strategies that codified the expertise of experienced journalists. It supported idea generation by presenting the retrieved news information and interactive sparks that codified creativity heuristics. The presented news information and sparks were designed to encourage individual journalists to use and learn simple creativity skills. INJECT was made available to an individual journalist as a single sidebar, so that the journalist could start news information searches and generate ideas about new angles next to a new story being written. As such, INJECT did not automate most of the journalist’s work. The journalist still needed to understand and select between recommendations to discover angles, as well as examine, interpret, style and present [26] new articles.

INJECT’s User-Centred Design
The interaction design of INJECT [18] involved journalists at all stages. After semi-structured interviews with journalism experts to discover the digital tools that journalists currently use, a decision was made to implement INJECT with Google Docs. Many journalists use Google Docs instead of bespoke news content management systems for early story development because of its familiarity, flexibility and higher number of useful features. To kick-start the user-centred design, different paper-based then digital wireframes of the sidebar were developed and presented to professional journalists. Over the next 8 months, new releases of the working INJECT software were tested for their usability and effectiveness, first with journalism students who had no direct relationship to the authors [18], then with professional journalists working in UK magazines, regional Norwegian newspapers and networks of freelance journalists in the Netherlands.

INJECT’s Sidebar: A Google Docs Add-on
To support journalists to undertake the pain-free exploration of information and ideas that was associated with creative thinking [10], INJECT’s digital creativity support was integrated into the Google Docs editor. Google supported a research sidebar of the editor window called the Google Docs Add-on that allowed a user to start searches and browse results [31]. The team decided to deliver INJECT’s creativity support through this Add-on, referred to simply as the sidebar in this paper.

The sidebar was the pre-defined Google Docs component that appeared to the right of the editor. However, the sidebar had a fixed width (300px) and poor performance with server-side libraries. Therefore, design challenges included embedding usable and effective creativity support that called different server-side services into the sidebar. To meet the challenges, INJECT was implemented with: (A) features to generate candidate search terms directly from text already written in the larger editor window; (B) small icons with mouse hover-box descriptions, to control the sidebar; (C) mouse hover-boxes to present additional information quickly in context; (D) overlays to present more information in the sidebar space, and; (E) server-side services that preloaded news information to overcome the sidebar’s performance limitations. The designs and implementations of these 5 features were improved incrementally during the journalist-led prototyping, for example by merging control icons and reducing the number of required interactions, so that, for example, the journalist received creativity support in as little as 2 clicks.

Figure 1 shows the final design of the INJECT sidebar. The left side shows the launch setup and 4 functional areas. The first area was the news topic space – a journalist could enter terms or import them directly from the text editor to describe topics of the current news story. The second area was the strategy space – a journalist could invoke different creative search strategies using the described topics. The third, which occupied over 80% of the sidebar, was the information space. A journalist could scroll, mouse hover-over and click retrieved news information and creative sparks to discover and generate new ideas for news angles. The fourth space, fixed at the bottom of the sidebar, was the control space. The journalist could access different tabs to manage tool settings, collaborate with online communities and follow a tutorial. An example of this tutorial is shown on the right side of Figure 1. A small number of INJECT features were also implemented in a separate dialog component that would appear over the editor window. However, due to the limited functionality of that component, most features were implemented in the sidebar, so that journalists could simultaneously use it and write new articles.

INJECT’s Creativity Support
Imagine a fictional journalist who used the INJECT Google Docs sidebar to discover and examine creative angles for a story related to the 2017 Italian migration crisis. At any time s/he could highlight text written in the editor – in this example: Europe’s annual summer migration crisis impacts one country more than any other – Italy – then click the insert button on the sidebar. INJECT parsed the highlighted text to extract stemmed nouns, verbs and proper names as candidate topics. The journalist could then edit these nouns, verbs and proper nouns before instructing INJECT to discover possible creative angles using topics such as migration, summer, crisis and Italy, shown in the news topic space at the top of the sidebar in Figure 2.

The journalist could then instruct INJECT to discover possible creative angles using 6 pre-defined types of angle by clicking icons in the strategy space. Each of the icons invoked a different creative strategy that searched news information sources associated with a different type of creative angle:

A. Quantifiable: discover and examine quantified information associated with the news story;

B. People: discover and examine information about people associated with the news story;

C. Causal: discover and examine information about events associated with the background of the news story;

D. Quirky: discover and examine comical information associated with the news story;

E. Ramifications: discover and examine information associated with future consequences of the news story;

F. Data visualizations: discover and examine data sets and visualizations associated with the news story.

Each of these 6 strategies codified the expertise that experienced journalists had reported to discover creative angles on news stories. To uncover and describe candidate strategies, semi-structured interviews with experienced journalists and digital experts in journalism had been held. The candidate strategies were then validated with other experienced journalists, and extended and refined [18]. Each strategy operationalized the elicited journalism expertise so that INJECT users could exploit it to discover possible creative angles on news stories.

So for example, if the fictional journalist clicked the people icon, INJECT used the selected terms to search and retrieve news information sources, extract the names of individuals from these source and present the extracted people’s names in the sidebar, with basic information about the individual and the original news item. Figure 2 also depicts the sidebar’s information space, in which one individual, Pope Francis, is presented above the information source in which he is named. The journalist could scroll the sidebar to search the information about Pope Francis and generate new ideas for story angles from it. Other individuals presented for this search were politicians Lulzim Basha and Vince Cable and philanthropist George Soros. Then, to examine a selected individual, the journalist clicked the individual’s name, and INJECT opened the dialog window with a photograph and description of the person extracted from Wikipedia sources, to stimulate more idea generation about new angles. If needed, the journalist could also request more people using the refresh button.

If the journalist clicked the causal icon, INJECT searched for and retrieved explainer-style, long-read news articles. It would present each article’s title, source and first line in the sidebar. Figure 3 depicts 2 examples of this sidebar, with an Independent newspaper article on an Amnesty International report on EU naval operations on the left side. Again, the journalist could scroll the sidebar to search presented information, generate new ideas for story angle from it, and request more information by clicking the refresh button. If s/he then clicked the article title, INJECT would open the article in the dialog window. This window then enabled the journal ist to search each article with further keywords and view the original article online – features designed to support idea generation – and to quick-reference the article and paste highlighted text from it into the Google Docs editor – features to support more productive work.

Clicking on ramifications would have triggered INJECT to use the topic terms to provide similar support based on articles that describe possible future implications.

Moreover, for each news article in the sidebar, INJECT also presented a set of entities that were extracted automatically from the article, in colored rectangles – green for people, red for events, brown for places and blue for organizations. These entities summarized each article’s content. When the journalist positioned the mouse over each rectangle, INJECT presented a pop-up creative spark generated for that entity that codified creativity heuristics specialized to the journalism task. The creative sparks were designed to encourage the journalism to use and learn simple creativity skills. The left side of Figure 3 shows one example of these sparks: Explore recent data that is available about Syria, and build on interesting patterns in it. Other sparks in this sidebar included: Explore the future implications for Italy and how might its people be impacted, and Explore the characteristics of human rights that enhance the emotional impact. Creative sparks were also displayed if the journalist placed the cursor over an article title or individual name in the sidebar, see the right side of Figure 3. The mouse hover-over feature enabled the journalist to explore multiple pieces of creative advice generated from retrieved news information quickly, consistent with design advice in [10].

Furthermore, if the journalist clicked any colored rectangle, the sidebar displayed an interactive concept card containing a text description of the entity. The journalist could learn from and incorporate the description into his/her article. The journalist could also edit the card’s content to maintain a more personalized set of concept descriptions with which to write future articles more productively. Figure 4 shows the sidebar when the journalist clicked on the Open Europe entity. The concept card was presented in the sidebar, over the original greyed-out news content that the journalist could return to with one click.

On the other hand, if the journalist clicked quirky, INJECT searched for and retrieved political cartoons with matching names and captions, and presented each retrieved cartoon as a thumbnail image and caption and creative spark. Clicking a cartoon thumbnail opened it in the dialog window to support further idea generation, as shown in Figure 5. If the journalist had clicked data visualizations, INJECT would provide similar support using data and information visualizations extracted from retrieved news sources.

In this example, our fictional journalist might have used the retrieved news information and creative sparks to generate a news story about the moral and legal conflicts between the roles played by the Papal State and Italian Republic. The story might also include the complexities arising from the intervention of non-governmental organizations in the Mediterranean Sea.

INJECT’s Architecture
To deliver the described interactive digital creativity support, the INJECT tool had 3 architecture layers:
– A user interaction layer that enabled different interfaces, such as the sidebar plug-in for Google Docs;

– A data layer of 1.6 million tagged news published news stories discovered using RSS feeds from 150 news sources. The sources were selected by INJECT’s journalist team to represent political perspectives and reduce the risk of echo chambers, and a database of over 40,000 political cartoons. INJECT also accessed information from Wikipedia but did not search it, so it was not part of the data layer;

– An application layer of software services that supported journalists to generate news stories more creatively and productively: (i) the creative search service manipulated topic descriptions from the editor to generate queries then implemented the different creative search strategies; (ii) the news extraction service collected and indexed information from the 150 news sources prior to creative search; (iii) the creative sparks service generated creative sparks that were tailored to entities extracted from news information; (iv) the concept card service that allowed individual journalists to edit and maintain personalised sets of concept cards, and; (v) the persistence service that provided search session storage and retrieval capabilities. The news extraction and creative sparks services pre-generated news information content for the sidebar, to reduce the impact of the Add-on’s performance constraints.

The news extraction service collected and indexed news information using public RSS feeds to the 150 news sources and tailored machine learning and natural language processing algorithms. It uploaded this information from the feeds every 30 minutes and stored it in a PostgreSQL database as metadata, with raw article data text as strings and a URL link to the source. It removed non-news content such as navigation links and adverts. It detected and extracted the people, location organization and event entities. It applied advanced natural language parsing to determine noun and verb phrases. And it uploaded each processed news article into an external Elasticsearch Cluster.

At run-time, in response to a journalist entering topic terms and clicking one of the creative angle icons, the creative search service retrieved relevant news information, in two stages:

1. It automatically disambiguated each topic term by discovering its correct sense according to the online lexicon at WordNet using context knowledge from other terms in the query (e.g. that migration is a group of people migrating together in some given time period rather than a periodic passage of groups of animals from one region to another for feeding or breeding) [20, 28]. It then expanded each term with other terms that have similar meanings (e.g. the term migration with the above sense is synonymous with terms such as relocation and exodus) and included these terms in the query. The service returned an unordered set of news articles or digital cartoons that achieved a threshold match score with the expanded search terms;

2. It filtered retrieved news articles and information using the strategy associated with the clicked icon. For example, for people, the service extracted from articles the name of each individual with a Wikipedia entry. For causal, it filtered to retain matched articles with more than 500 words and a minimum threshold of keywords indicative of causal articles – terms such as cause, impact and studies – from sources such as the Economist and the New York Times. And for quantifiable, it filtered to retain articles with a minimum threshold of quantity, measure and value keywords, for example Sterling, population and actual numbers.

The creative sparks service generated the pop-up sparks for each retrieved article and entity extracted from each article. An individual creative spark associated 1 extracted entity or news article to 1 creative instruction. The sets of instructions had been manually generated from websites and blogs that teach journalists to uncover new angles on stories. One set of instructions was generated for each of the 4 types of entity that were extracted – people, events, places and organizations. Examples included: Unpick what the relevance of [Place], as opposed to somewhere else, might have on the story and; Explore the history and background of [Organization] to obtain a new perspective on your story. A total of 34 creative instructions were implemented. One set of instructions was also generated for news articles retrieved with each of the 6 creative strategies – people, causal, quirky, quantifiable, ramifications and data visualizations, and a total of 41 such instructions were implemented. Examples included: Use data types reported in this story, to generate a new angle, and: Make your angle more similar to the causal angle in this story. When invoked, the service used a randomizing function to attribute one instruction string to one entity string of the same type, then concatenated the strings to generate the spark. So, for the extracted organization Open Europe from Figure 4, INJECT might have presented a spark such as: Explore the history and background of Open Europe to obtain a new perspective on your story.

To retrieve and manage the concept cards, INJECT incorporated Explaain, a service that organized news information in forms other than in articles [8]. Each concept card normally stored one small chunk of news-related information, often similar to the length of a tweet. Phrases in each card were linked to other cards, similar to hyperlinks.

INJECT also implemented other features that were requested by journalists during the prototyping to:

– Discover news information and provide creative guidance in the languages that journalists work in, for example in Dutch and Norwegian as well as English;

– Discover information from newspaper archive sources, for example the digital archives of participating regional Norwegian newspapers.

Although the design of INJECT had involved journalists, there had been no testing of the tool in newsrooms to explore its impact on journalist work. The next section reports first evaluations of the impact of INJECT on journalist’s work using an established journalism framework.

Evaluating INJECT in newsrooms

A version of INJECT that implemented the people, causal, quantitative and quirky strategies and all features reported in previous sections was used by 5 journalists to develop published news stories. Data collected after this use was analyzed to answer the following 2 research questions:

RQ1: Did each journalist use INJECT to discover new and useful angles on news stories?

RQ2: Did each journalist change how s/he developed news stories as a result of using INJECT?

Evaluation Methods
The INJECT version was evaluated by 5 journalists. Three of the journalists worked for 3 different regional Norwegian newspapers from the INJECT consortium and were selected by their editors. The 3 journalists were: an all-round journalist responsible for content in the online version of the newspaper, with 4 years of journalism experience; a photojournalist who produced video reports, with 2 years experience, and; a news journalist who decided the content of all newspaper versions, with 13 years experience. One of the 3 journalists had limited involvement in the earlier codesign of INJECT. The other 2 journalists who evaluated INJECT worked at the same UK newspaper that specialized in digital journalism, and were previously unconnected to the project. One was an editor with 3 years experience, and other a senior reporter with 2 years of experience. Other journalists also used INJECT, but the lack of time and resources in newsrooms that INJECT sought to alleviate also reduced the opportunities to provide evaluation feedback.

Each journalist agreed to use INJECT in his/her work and to be interviewed afterwards about this use. No other incentives were offered. After agreeing to participate, each journalist received an email with instructions to download and use the INJECT sidebar. Each was also sent links to an INJECT help website and was able to ask questions to the research team, to become familiar and competent with the tool’s features. The 3 Norwegian reporters used a Norwegian-language version of INJECT that also searched local newspaper archives, a screenshot of which is shown in Figure 6. All other INJECT capabilities were the same. When each journalist was familiar with the tool, his or her individual evaluation began. Two methods were used to collect data, depending on the amount of time that each journalist was able to commit to the evaluation.

The first method was simpler and used with the 3 Norwegian journalists. One semi-structured interview with each journalist was held in Norwegian after up to 4 weeks of INJECT use. The interviews asked how the journalist perceived and used the tool, and its advantages and disadvantages for discovering new news angles.

The second method was more complex and used with the 2 journalists able to commit more time. It collected two types of data from each journalist over iterations of tool use: (1) the news stories developed with INJECT’s support, and; (2) responses to structured interview questions. The researcher prepared for each interview with each journalist by first reading the news stories and adapting the questions, which were derived from Shapiro’s assessment framework for the practice of journalism [26]. The framework defined 5 faculties of good journalism: discovery, examination, interpretation, style and presentation. INJECT was intended to support the discovery and examination faculties. Discovering was the faculty of deciding what to say, and uncovering the news angle from information sources. Examining was the faculty of testing facts from information sources for their verifiability and coherence. The interview questions were derived from quality-related attributes associated with these 2 faculties. To elicit evidence about discovering, the interviewer asked questions about INJECT’s influence to: (a) select the stance to investigate the topic; (b) overcome journalist topic biases and interests, in order to achieve pragmatic objectivity; (c) ensure social importance, to leave society in a better place than before with more socially important stories, and; (d) be ambitious in the methods used by the journalist [26]. To elicit evidence about examining, the interviewer asked questions about: (e) use of rigorous efforts to ensure accuracy, and how INJECT use might have increased rigor and accuracy, and; (f) being undaunted in the research to shed new light on complex subjects [26].

First Evaluation Results
The one-off semi-structured interviews with the 3 journalists at the 3 Norwegian newspapers revealed that INJECT was used to support the development of 5 published news stories – 3 by 1 journalist and 1 each by the other 2. All 3 journalists used INJECT to develop stories that were assigned to them after editorial meetings. All also reported minor usability problems, such as unclear meaning of the sidebar’s 6 creative search icons, and the unnecessary clicks needed to change INJECT’s creative search from strict to relaxed. However, none of these problems impeded the journalist’s use of INJECT.

All 3 journalists reported that INJECT use supported them to discover new news angles, at the time that stories needed to be written, and to include more background information in their stories using the quantifiable strategy. INJECT use was reported to be more effective for discovering new angles on longer feature stories that required evidence, numbers and facts, rather than on shorter news stories that reported facts. This decision to use INJECT for occasional feature stories explained the low number of articles that were written with INJECT’s support. In addition the 3 journalists also reported that INJECT use supported the discovery of information from their own newspaper archives – information that each already should have remembered. And all 3 reported that writing stories more effectively with angles that were generated by themselves, which INJECT supported them to do, rather than with automatically generated angles that the tool recommended to them.

The new ideas and angles on the news stories were discovered by all 3 journalists using INJECT quickly, often in less than 3 minutes for each story. The journalists then switched to other tools, such as Google search, to retrieve more detailed information with which to complete the story.

No direct disadvantages of INJECT use were reported, but the journalists expressed reservations about some of its design. The journalists requested more explanation of the creative search service algorithms and news information that they retrieved. None claimed to use the creative sparks, and 1 journalist asked for the sparks to be presented in different forms, to standout from other information in the sidebar. One journalist also asked for digital reminders of other established journalism angles, such as to think about the economic angle on a story. Other reported new requirements were integration into the newspaper’s content management systems, and a critical mass of 4-5 journalists who would use INJECT regularly in each newsroom, to maintain enthusiasm for and training and support to use the tool.

To summarize, the semi-structured interviews revealed that journalists did use INJECT to discover new angles on news features before these features were written, often quickly, before using other digital tools to complete the stories. A critical mass of journalists using INJECT in newsrooms was requested to support ongoing use of the tool. However, the interviews revealed little about how the individual journalists changed their behaviour as a result of using INJECT.

Second Evaluation Results
A total of 4 interviews took place. The editor and senior reported that 4 published news stories were written with support from INJECT, and other stories had been prepared with use of the tool. The senior reporter claimed that the permanent presence of INJECT’s sidebar changed how she wrote the news stories. This presence led her to use its features after writing each paragraph rather than the end of the story – a change that she believed decreased the chances of omissions from stories. She also reported that she perceived INJECT’s strategies as alternative forms of journalism’s 5 Ws (who, what, where, when and why) that constituted a formula for developing a complete news story on a subject, and that INJECT’s quirky and ramifications strategies extended this formula in new directions. The editor reported using the creative search icons from left-to-right on the sidebar, in order to structure her exploration of new angles with INJECT.

The timing of INJECT use impacted on its effectiveness. At first, the editor and senior reporter used INJECT only after having developed the structure and some of the text of their stories, equivalent to the senior reporter’s use of Google search, and both were disappointed by the wide range of the articles presented by INJECT. Later in the evaluation period, both used INJECT earlier: the editor used it during the research stage of a new article on copyright law, to investigate different topics entered into the sidebar. She reported that the information was still “vague .. gave me articles that were not quite what she was looking for”, although while this information did not change the overall direction of the article, it did guide her to incorporate new voices into it. The senior reporter used INJECT to research of a new story about news fact-checking, and made use of its quantifiable and causal strategies to explore new angles.

In both interviews, the editor reported that INJECT use led her to discover articles that informed her stories from beyond her usual news sources. Examples of these new news sources were Wired magazine, Guardian newspaper, blogs, digital cartoons and other newspapers about the topic but from 4-5 months earlier. INJECT’s use appeared to overcome some of her biases arising from sources used. In particular she reported that news information retrieved with the people strategy led her to introduce new voices from alternative sources into the article about copyright law. She also reported that one creative spark – think about the opponents of [Person], and how the story evolves from their perspective – directed her research in that direction and to generate new content to the article. On reflection, the senior reporter claimed that INJECT’s less directed searches had the potential to develop new stories with greater social importance, saying: “it is a very good tool for inspiration”. To this end she used the people strategy to develop one story from the perspective of people who were impacted by fake news, rather than from the data sources that she usually used. She highlighted the potential of the ramifications strategy to encourage her to explore more about the consequences of events, rather than just report the events themselves.

The senior reporter stated that INJECT had the potential to increase the boldness of her journalism methods by recommending new angles on her own already-implemented stories. These new angles guided her to repurpose and publish new versions of these stories, which was a new direction for her journalism. The editor reported that INJECT also had the potential to increase the boldness of her journalism, in part due to the authority that she attributed to news stories retrieved by the INJECT tool compared with her searches of social media sources. She also claimed that the creative sparks had the potential to increase the social importance of her stories, because some of the sparks led her to generate new ideas about other socially relevant organizations.

INJECT’s creative search strategies were reported to have changed the behaviour of both journalists, albeit less directly than had been designed for. For example, the existence of the people and quirky strategies made the editor more aware of these types of angle: “it made me search a bit on the individual and the quirky side. Cos I think that I wouldn’t have thought to look for cartoons…”. And the senior reporter claimed that the presence of INJECT’s quantifiable and ramifications strategies reminded her to think more about the accuracy and the future implications of her stories, both with and without INJECT.

The editor reported that, on reflection, she understood better INJECT’s purpose and potential benefits to her work. These benefits included greater support for writing feature stories, use of creative sparks to direct her to think of more people, and to be more ambitious in both her stories and investigative methods. About the creative sparks, the editor said: “some of them are very, very specific, and some are a bit more broad, so in that sense it could potentially give you a bit of a push. So if you are writing a story and you’re not sure on the angle, and if you should include this or this person in it, then if you go to the prompts, then some of them might resonate, so yes..”.

Neither journalist reported strong evidence that INJECT supported examining news information that was retrieved, but the senior reporter said that her use of the quantifiable strategy did provide her with some data and evidence that supported the examining faculty. However, the editor stated that INJECT was not an effective tool for fact checking.

Conclusion, discussion and future work

This paper reports the design and first evaluations of new digital support for journalists to discover creative angles on news stories. The digital support integrated natural language processing, creative search and interactive recommendation technologies. It codified the expertise of experienced journalists to retrieve news information and direct idea generation, consistent with human-centred creative cognition [15]. The support was delivered to journalists as a visible sidebar of a text editor. Data was collected from 5 journalists to answer 2 research questions:

RQ1: All of the journalists discovered new and useful angles on stories written with INJECT’s support, often quickly, however many of these angles extended existing storylines rather than developed new ones;

RQ2: Some changed how they developed news stories as a result of using INJECT, and the presence and structure of the INJECT sidebar in the editor window impacted on the behaviour of at least some of the journalists.

As such, journalist use of INJECT was effective, but not to the degree that the tool was designed to support. Journalist use of INJECT also provided more effective support for writing feature stories rather than news stories, although this finding might have been influenced by INJECT’s data layer, which was only composed of past news rather than current social media information. However, writing feature stories will still require journalists to exhibit more creative thinking than during the evaluations, by applying more creativity skills when using INJECT than was reported.

Results revealed that use of INJECT offered more support to journalists when discovering rather than examining [26]. It appeared to contribute to journalists being more ambitious and to overcome biases by retrieving topic information from alternative news sources and discovering new stances from which to investigate news topics. However, this support for discovering also led journalists to compare INJECT, sometimes unfavorably, to Google search, a comparison reinforced by some of the journalists using Google search after INJECT to seek more information on new angles. At least some of the journalists might have expected INJECT to discover the information that was needed, rather than just to offer creative support with this information, as INJECT was designed to do. Although subsequent versions of INJECT now embed Google search to support journalists, to help to distinguish between discovering a new angle with INJECT capabilities and retrieving more information afterwards, journalists will still need to apply more creativity skills to exploit INJECT’s capabilities effectively. The reported uses of INJECT did not appear to deliver and/or encourage sufficient use of these skills.

Results also revealed that use of INJECT supported some of the journalists to recognize known news information at the time that news stories are being written. Indeed, different news organizations have identified the potential of INJECT to unlock their own news archives for creative use, to enhance productivity as well as creativity, but only if INJECT can support journalists to recognize [5] related news quickly when writing stories. New recommendation features will be needed to support such recognition.

Of course, many threats to the validity of these first evaluation results exist. Although more journalists used INJECT, only 5 provided structured feedback for the evaluation, and their verbal reports were not triangulated with INJECT usage log data. The journalists worked for regional and specialist newspapers rather than other types, and did not use INJECT support for most of their news stories. The regional newspapers were members of the project consortium and their journalist’s responses might have biased, due to pressure from management and/or loyalty. And there has been no systematic evaluation of the creativity, i.e. novelty and value of the news stories generated with INJECT’s support. Therefore, the results need to be interpreted cautiously, and more systematic evaluations of INJECT with larger numbers of journalists for longer periods will be needed to draw firmer conclusions about the tool’s effectiveness.

Journalists will need to demonstrate more creativity skills to exploit INJECT effectively. Some of the journalists in the evaluation expressed a preference to write stories based on ideas generated themselves rather than ones recommended by INJECT – a preference can be interpreted as motivation to be more creative and to develop and apply creativity skills. However, development of the skills remains a challenge in newsrooms with conservative attitudes [7], especially in the face of new fears over automation that INJECT itself might exacerbate. For example, the tool could be extended to collect data about the dates and times that readers access stories generated with different INJECT angles, to automate the writing of stories with different angles to be read at different times. Therefore, although future versions of INJECT will include more directed creativity support, digital tools alone will not provide the missing creativity skills. One alternative to explore with newsrooms is upfront journalist training in creativity techniques that align with INJECT’s capabilities. One example of these techniques is SCAMPER [22], which guides problem solvers to examine solutions from different perspectives such as combining, eliminating and reversing, and is a simplified form of INJECT’s creative sparks. Another example is Hall of Fame [22], a technique that guides problem solvers to examine solutions from the perspective of different well-known people – who could be iconic journalists. We hope to report on the use of these techniques and INJECT together in the near future.

Another possible way to develop and apply creativity skills is to develop a critical mass of journalists using INJECT in each newsroom, as some of the journalists in the evaluations reported. It is long established that during idea generation and evaluation activities, cooperation can lead to more creative ideas and better solutions (e.g. [32]), and cooperation between 4-5 journalists has the potential to support the development and use of different creativity skills. When journalists used INJECT also appeared to influence the effectiveness of the tool. Therefore, to encourage earlier use of INJECT and the new creativity skills of journalists, we will seek to embed this use into newsroom workflows, for example for short periods after editorial meetings and prior to writing news stories.
Another emerging challenge was to design complex interactions that inputted to and received outputs from sophisticated algorithms that produced unpredictable outcomes. Artificial intelligence has been framed as a new design material that required interactions that were transparent, opaque and offer shared control [12], and journalists both in the evaluations and elsewhere requested transparent creative search algorithms to explain the retrieved news information. However, INJECT’s designers also believed that these algorithms still needed to be opaque, in order to retrieve information that was surprising and hence creative to users. Therefore, future work will refine INJECT’s interactions to offer different degrees of support related to the user’s familiarity with the tool, for example to offer greater transparency and shared control to less experienced users, and to increase the opaqueness for more experienced ones.

Other new INJECT features have already been implemented. These features include more news and new social media information sources added to the data layer, contextualized explanations of information retrieved by INJECT, new uses of the creative sparks, and new versions of INJECT for the InCopy TinyMCE text editors. Future uses of these new INJECT versions by critical numbers of journalists in newsrooms will enable the research team to investigate broader research questions about changing journalism practices. For example, future versions of INJECT will invoke fact-checkers to detect possible fake news information. However, if journalists do develop and use new creativity skills with INJECT, the resulting creative thinking might also increase their reflexivity with which to detect fake news sources. More broadly, use of new digital tools such as INJECT have the potential to increase conflicts in newsrooms [7], with journalists who resist the adoption of these new tools, and with managers who impose the new tools in order to increase newsroom creativity and productivity. Moreover, as newsroom resources reduce further, uses of more intelligent tools such as INJECT risk substituting rather than supplementing key journalism tasks such as face-to-face interviewing. Therefore, the use of new digital tools to increase journalist creativity cannot be separated from wider socio-political challenges that newsrooms face. The rollout of INJECT in new newsrooms will be a vehicle to investigate emerging challenges and trade-offs that will we anticipate will arise.

Acknowledgements

The research reported in this paper is supported by the EU-funded H2020 723328 INJECT innovation action.

References

1. Alaou, S. F., Schlphorst, T., Cuykendall, S., Carlson, K., Studd, K. and Bradley, K. 2015. Strategies for Embodied Design: The Value and Challenges of Observing Movement. In Proceedings of the ACM Conference on Creativity and Cognition (C&C’15), 121-130. http://doi.acm.org/ 10.1145/2757226.2757238

2. Amabile, T. M. and Pratt, M. G. 2016. The Dynamic Componential Model of Creativity and Innovation in Organizations: Making Progress, Making Meaning. Research in Organizational Behavior 36: 157–183.

3. What is Journalism? Retrieved December 28, 2017 from https://www.americanpressinstitute.org/journalism-essentials/what-is-journalism/

4. Bartindale, T., Valentine, E., Glancy, M., Kirk, D., Wright, P and Olivier, P. 2013. Facilitating TV Production Using StoryCrate. In Proceedings of the ACM Conference on Creativity and Cognition (C&C’13), 193-202. http://doi.acm.org/ 10.1145/2466627.2466628

5. Baddeley, A. 1982. Your Memory: A User’s Guide. Multime-dia Publications.

6. Currah, A. 2009. What’s Happening To Our News? Reuters Institute for the Study of Journalism, Oxford.

7. Ekdale, B., Singer, J. B., Tully, M. and Harmsen, S. 2015. Making Change: Diffusion of Technological, Relational, and Cultural Innovation in the Newsroom. Journalism and Mass Communication Quarterly 92,4: 938-958.

8. Bring your articles to life. Retrieved December 28, 2017 from http://explaain.com

9. Garbett, A., Comber, R., Egglestone, P., Glancy, M. and Olivier, P. 2014. Finding “real people”: trust and diversity in the interface between professional and citizen journalists, In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI’14), 3015-3024. http://doi.acm.org/10.1145/2556288.2557114

10. Greene, S. L. 2002. Characteristics of Applications that Support Creativity. Communications of the ACM 45,10: 100-104. http://doi.acm.org/10.1145/570907.570941

11. Gynnild, A. 2014. Journalism innovation leads to innovation journalism: The impact of computational exploration on changing mindsets. Journalism 15,6: 713–730.

12. Holmquist, L. 2017. Intelligence on Tap: Artificial Intelligence as a New Design Material. ACM Interactions 24,4, 29-33.

13. Honauer, M. and Hornecker, E. 2015. Challenges for Creating and Staging Interactive Costumes for the Theatre Stage. In Proceedings of the ACM Conference on Creativity and Cognition (C&C’15), 13-22. http://doi.acm.org/ 10.1145/2757226.2757242

14. Kaufman, J. C. and Beghetto, R. A. 2009. Beyond Big and Little: The Four c-model of Creativity. Review of General Psychology 13,1.

15. Kerne, A., Koh, E., Smith, S. M., Webb, A. and Dworaczyk, B. 2008. combinFormation: Mixed-Initiative Composition of Image and Text Surrogates Promotes Information Discovery. ACM Transactions on Information Systems 27,1: 1-45.
http://doi.acm.org/10.1145/1416950.1416955

16. Liaw, R., Zilnik, A., Baldwin, M. and Butler S., S. 2013. Maater: crowdsourcing to improve online journalism. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI ‘13), 2549-2554. http://doi.acm.org/10.1145/2468356.2468828

17. Maiden, N., Zachos, K., Lockerbie, J., Levis, S., Camargo, K., Hoddy, S. and Allemandi, G. 2017. Evaluating Digital Creativity Support To Improve Health-and-Safety in a Manufacturing Plant. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI ‘17), 7005-7014.
http://doi.acm.org/10.1145/3025453.3025712

18. Maiden, N., Zachos, K., Lockerbie, J., Brock, G. and Traver, C. 2016. Developing and Evaluating Digital Creativity Support in Google Docs for Journalists. In Proceedings of the 30th International British Human Computer Interaction Conference (HCI’16), Article No. 23. http://doi.acm.org/10.14236/ewic/HCI2016.23

19. Malmelin, N. and Virta, S. 2016. Managing creativity in change: Motivations and constraints of creative work in a media organization. Journalism Practice 10,6: https://1041-1054. https://doi.org/10.1080/17512786.2015.1074864.

20. McCarthy, D., Koeling, R., Weeds, J. and Carroll, J. 2004. Using Automatically Acquired Predominant Senses for Word Sense Disambiguation, In Proceedings of the ACL 2004 Senseval-3 Workshop. Association of Computational Linguistics.

21. McNair, B. 1998. The Sociology of Journalism. London: Arnold.

22. Michalko, M. 2006. Thinkertoys: A Handbook of Creative-Thinking Techniques. Random House Inc., New York

23. Minard, A., Speranza, M., Agirre, E., Aldabe, I., van Erp, M., Magnini, B., Rigau, G. and Urizar, R. 2015. SemEval-2015 Task 4: Timeline: cross-document event ordering. In Proceedings of the 9th International Workshop on Semantic Evaluation, 778-786.

24. Sharing Liberally. Retrieved December 28, 2017, from http://bostonreview.net/BR35.4/morozov.php

25. Schofield, T., Vines, J., Higham, T., Carter, E., Atken, M. and Golding, A. 2013. Trigger Shift: Participatory Design of an Augmented Theatrical Performance with Young People. In Proceedings of the ACM Conference on Creativity and Cognition (C&C’13), 203-212.
http://doi.acm.org/10.1145/2466627.2466640

26. Shapiro, I. 2010. Evaluating Journalism. Journalism Practice 4,2: 143,163.

27. Sternberg, R. J. (Ed.). 1999. Handbook of Creativity. New York, Cambridge University Press.

28. Stevenson, M. and Wilks, Y. 2001. The Interaction of Knowledge Sources in Word Sense Disambiguation. Computational Linguistics, 27,3: 321-349.

29. jøvaag, H. 2014. Homogenisation or Differentiation? The Effects of Consolidation in the Regional Newspaper Market. Journalism Studies 15,5: 511-521.

30. Van Der Haak, B., Parks, M. and Castells, M. 2012. The Future of Journalism: Networked Journalism. International Journalism of Communication 6: 2923-2938.

31. Google Docs introduces new sidebar research tool. Retrieved December 28, 2017 from https://thenextweb.com/apps/2012 /05/15/useful-google-docs-introduces-new-sidebar-research-tool

32. Warr, A. and O’Neill, E. 2005. Understanding design as a social creative process. In Proceedings of the ACM Creativity and Cognition Conference (C&C’05), 118-127. http://doi.acm.org/10.1145/1056224.1056242

33. Witschge, T. and Nygren, G. 2009. Journalism: a profession under pressure? Journal of Media Business Studies 6,1: 37-59.