You know the importance of technology to the future of journalism has become a widely accepted fact when a prominent editor decides to join a new company because of its content management system. That’s what Ezra Klein told The New York Times about his decision to leave The Washington Post for Vox Media, a digital publisher with a fancy, custom-built CMS. Klein couldn’t quite describe what made the Vox system so special, but the fact that a journalist said he loved, let alone even tolerated, his CMS was all you needed to know that the world has changed.
Suddenly, the CMS, an often derided but necessary tool of modern journalism, is cool. Vox uses its CMS as a recruiting tool. Google is not-so-secretly building a CMS for the news industry. Times media columnist David Carr recently devoted an entire column to the up-and-coming blogging platform/CMS called Medium, and proclaimed that “the content management system is destiny.”
We couldn’t agree more. Here at The Times, our own CMS, Scoop, is central to our ambitions to innovate on all platforms. It’s also the repository for all the aspirations for what the merging of print and digital journalism may one day become — and many of the frustrations for what it is today.
With that in mind, we thought it was a good time to take a closer look at Scoop’s past, present and future — what it does well, what can be improved and how it will help The Times remain the finest journalistic organization in the world.
What is Scoop?
Scoop (not to be confused with our mobile listings app, The Scoop) is The New York Times’s homegrown digital and (soon-to-be) print CMS. (We also use WordPress for many of our blogs.) Scoop was initially designed and developed in 2008 in close partnership with the newsroom. Unlike many commercial systems, Scoop does not render our website or provide community tools to our readers. Rather, it is a system for managing content and publishing data so that other applications can render the content across our platforms. This separation of functions gives development teams at The Times the freedom to build solutions on top of that data independently, allowing us to move faster than if Scoop were one monolithic system. For example, our commenting platform and recommendations engine integrate with Scoop but remain separate applications.
This post was written and edited in Scoop.
The vision for Scoop has evolved over the years. The beauty of a homegrown CMS is that we can shape its features and technology over time. Since its inception, the Scoop platform has been extended to include many new features such as sophisticated authoring and editing tools and workflows, budgeting, photo manipulation, video management and more robust content APIs. Its user base has swelled from a few dozen web producers to more than 1,000 users, including reporters, copy editors, photo editors and video producers.
Perhaps the biggest change has been the reversal of our publishing process. The original idea was that articles would be written in the Microsoft Word-based print system, CCI, and then sent to Scoop, where a web producer would add multimedia, tag the content and publish it on NYTimes.com. Today, instead of writing articles in CCI and then sending them to Scoop, our journalists can create articles in Scoop and publish to web and mobile first before sending them to CCI for the print newspaper. We call this change “Digital First” — a multiyear project that will make Scoop the primary CMS for both print and digital by 2015.
It’s a powerful system, custom-built for the needs of a 24/7 print, web and mobile news organization. Scoop publishes roughly 700 articles, 600 images, 14 slide shows and 50 videos per day. With that power comes significant complexity, especially in comparison to the content management systems of our digital-only competitors. Unlike many simpler systems where a journalist can single-handedly take an article all the way from inception to publication, Scoop is role-based to reflect the segmentation of duties and the checks and balances in our newsroom. Managing that complexity — making the system as easy to use as possible without diminishing its features and functionality — is a balancing act we face every day.
Here’s a look at a selection of Scoop’s features and what we’re planning for the future.
Story Budgeting and Planning
Once upon a time, editors used to plan out the print edition of The Times by printing out story lists and circulating them among the various desks in the newsroom. Many trees were killed in this archaic and inefficient process. A few years ago, we built a separate tool to bring all planning online and then integrated that tool into Scoop. Editors still bring a lot of paper to Page One meetings, but online budgeting makes managing a global news operation of multiple desks and bureaus possible in a digital age. It’s also often the first step before an article or other type of content is created in Scoop. We plan to extend this functionality into a system that knows the status of each article, when it is will run online and in print, what multimedia will run with each article and how that article is performing once it is published.
Track Changes and Comments
One of the most distinguishing features of Scoop is its ability to do in-line track changes similar to what you would find in Microsoft Word. Many other browser-based text editors, such as Google Docs, show changes
in the margin. Our editors prefer (well, demand) Word-style track changes for text editing, not a diff engine that compares the current text to previous versions. This style of track changes dates back to the earliest
computer text editors such as Atex, which were the standard in newsrooms in the 1980s, and the style definitely has its merits by showing all edits in context. At our editors’ insistence, we also provide
in-line comments and notes rather than notes in the margin. To accomplish this, we built our own text editor called ICE, and then open sourced it.
Drafts and Workflow
Our editors can create unlimited drafts of articles. Many content management systems support a published version and a single draft, but in Scoop, we can have a live version of an article, a version where minor edits are being made, a version where the story is going through major updates and a print-specific version edited to fit in the paper. Each of those drafts can go through our editing workflow steps independently and then become the published version of that article.
Real-Time Collaboration
We lock content so that editors can’t overwrite each other’s work. Scoop allows editors to “knock,” which notifies the person with the lock on the content and gives an option to release that lock for the requesting user.
Today, we lock groups of fields together. For example, a reporter can work on the article while an editor is writing the headline and summary and a producer is adding multimedia. But one editor can’t work on the headline while another works on the summary.
We will soon release a new article user interface that will include field-level locking and real-time collaboration. In this new UI, editors will be able to see who else is working on the article and which fields each
editor has locked. When an editor saves a change, it is updated for all the other collaborators in real time. (At this point, at our editors’ request, we are not showing every keystroke each collaborator
is making as Google Docs does, but with a minor implementation tweak we could, and we are also exploring ways to have more than one person work on the text of an article at the same time.)
Private Articles
When working with highly sensitive material, our editors can mark an article as private and restrict access by user or group. This is a necessary requirement for an organization investigating the National Security Agency, for example, or publishing blockbuster op-eds by Vladimir Putin or Angelina Jolie.
Tagging Content
The Times has been categorizing content based on subjects, people, organizations and places since 1851. Today, tags are automatically suggested based on the content of the article using sophisticated algorithms that can tell the difference between “Georgia” the state and “Georgia” the country. Editors use those suggestions to tag the article. They can also manually select additional tags from the taxonomy or request that a new term be added. Our digital taxonomy team evaluates those requests, maintains the suggestion rules, adds new terms and reviews how articles are tagged on a daily basis.
Notifications
Scoop has a notification system that allows users to customize alerts that will let them know when articles matching custom criteria have been created, reached a certain workflow state, been updated or been published. This system can help prevent some editors from standing over the shoulders of harried reporters writing on deadline. It also helps producers who may want to know when every story in their sections are ready to be published, or copy editors who want to know when a particular story is ready for review. Notifications are sent via email or through Scoop itself.
Photos
When Scoop launched, our photo editors and producers only needed to crop a few different aspect ratios and sizes for each image. As the number of apps and platforms that published our content grew, so did the number of image crops we needed to provide. We didn’t want to change the newsroom’s workflow each time we launched a new app or the resolution on the iPad changed, so we revamped our image cropping tools to simplify the cropping process while increasing the number of sizes and aspect ratios that we publish for each photo. Today, editors use Scoop to draw the “master” crop of the photo and the “small square” or thumbnail of the photo. Scoop uses that information to make educated guesses on the rest that can be refined by an editor.
Make a few cropping decisions in Scoop:
And it will create 50+ images in a wide range of sizes and aspect ratios:
Adding Multimedia to an Article
Images, slide shows, videos, charts and graphics can be generally associated with an article, or placed between specific paragraphs. If an item isn’t specifically placed, NYTimes.com is programmed to “sprinkle” the multimedia into the body text to avoid collisions with placed images and ads.
Auto-Suggesting Content
Scoop uses APIs provided by our search team to suggest videos from our archive that are likely relevant to the article being produced.
Web and Mobile Preview
Scoop has always supported a pixel-perfect preview of what unpublished content will look like on NYTimes.com. We recently added a panel to that preview screen that also shows how the article will render on our mobile site.
Content APIs
Scoop’s publishing job is not done until it has provided APIs to power our web and mobile platforms. This is an area that has significantly changed over the years as the number of platforms we support has grown. In January, we launched a new publishing API named PAPI to support the updated site architecture that came with our site redesign.
Here’s what’s next for Scoop.
Simplifying Print Workflow
It was a huge leap forward when articles written in Scoop could be sent to our print layout systems. But this can create workflow problems as stories continue to develop late in the day. Keeping articles up to date both in Scoop and our print system is problematic. To solve this, we are working on a project to allow our editors to copy-fit for print directly from Scoop. Once that is completed, all print content will be edited in Scoop, while page design will continue to take place in the print system.
Concentrating on Reporters
Scoop is designed around workflows where a reporter writes text, a copy editor writes a headline and summary, a photo editor selects the images and a producer puts it all together. This describes how most of our report is currently assembled, but we don’t think this is necessarily the model for the future. Scoop does not do a good job of accommodating writers who want to do all of those things themselves to tell a story. Nor is it a tool for a reporter who only wants to write and wants the simplest, most elegant interface possible to accomplish that task. Working with our newsroom, we are exploring ways in which we can have the best of both worlds — an elegant user interface for writers and (with the click of a button) a powerful editing and workflow management tool for editors and producers.
Integrated Metrics
The Times has many talented people investigating traffic and social media trends for each piece of our content. At the newsroom’s request, we have never integrated that data back into Scoop to help inform our editors on how to best reach our readers. That attitude is shifting in our organization, and we plan to explore new ways to display this type of information for our editors.
Enhanced Read/Write APIs
We are a large technology organization that includes a team of engineers in the newsroom working directly with editors and reporters to cover the news. Our graphics desk also includes engineers and data visualization journalists who create interactive charts and timelines that readers regularly see in our daily coverage. Scoop provides some read/write APIs to contribute to this effort, so that those tools can write directly to Scoop where they can be associated with articles and published. This interaction isn’t as smooth as it should be, and we are working together with those teams to make Scoop and these external tools feel like a cohesive unit to our newsroom. This should simplify the tools and result in even more interactive pieces appearing in our coverage.
Rethinking Section Fronts for Mobile and Beyond
The NYTimes.com home page and most of our section fronts are manually curated so that our readers can tell which stories our editors have deemed the most important throughout the day. This curation powers the sections of our apps and mobile site as well. The manner in which section fronts are “ranked” is tied to the desktop website’s layout, which means we then need to translate those layout decisions to our apps and mobile site where the layouts are different. With the majority of our traffic shifting to mobile devices, we want to give our editors the same level of control over the presentation of content in mobile that they have today on the web. We are re-evaluating our current approach to come up with a less labor-intensive way to convey the importance of each piece of content, and to structure the data in a way that can be more directly applied in different layouts on phones and tablets. This will require some significant changes to Scoop, our mobile apps and the newsroom workflow. The good news is that we have a system that we control and a team more than capable of making those changes.