Avoiding Technical Debt – A Scholarly Publishing Imperative

Almost every scholarly communications industry market report cites technology as one of our greatest challenges. In fact, every industry – even politics, as the world has become painfully aware – is facing technological challenges in terms of keeping up with maximizing productivity, maintaining security, supporting ecommerce, refining discovery, and so on.

Having recently completed several studies related to publishing workflow technologies, I found it interesting that supporting technologies are basically broken up into three groups. The lines can blur, but platforms, tools, and systems supporting the publishing workflow (not including back office functions like accounting and customer management systems) tend to fall into one of three categories:

• The Back End or Intake. This includes:
  • Authoring tools
  • Manuscript submission platforms (which includes ecommerce if Open Access)
  • Peer review systems
  • Rights management systems
• The Middle or Management:
  • Production systems
  • Enterprise content management systems (CMS or ECMS)
• The Front End or Delivery:
  • Publishing platforms that disseminate digital content (including search functionality) and support ecommerce

Established vendors across the workflow tend to segment their offerings into these three areas as well, with modules or pre-built functionality that can be utilized to support most – but typically not all – publisher requirements for any part of the workflow.

Interviews with a small number of mid-sized publishers revealed the following:

• All are making significant technological changes, either building new systems internally, working with vendors, or both;
• The majority are building internally because off-the-shelf solutions don’t fit every need, and in some cases don’t meet most needs without significant customization;
• None have a system that serves all functions. In other words, none use single systems for each of their workflow functions - front, middle or back;
• All are working on drafting development roadmaps; none has a clear path to where they want to be when system construction is complete;
• None with are 100% satisfied with vendors supporting some workflow functions, primarily because their technologies are dated or limited, which is one reason they’re looking to replace them;
• A driving factor for working with a specific vendor is not only functionality but customer service: Relationships drive many technology choices.

These findings point to a pressing issue facing most content providers today: avoiding technical debt (also called design debt or code debt). Because the cost of continually upgrading technology can be both operationally and financially daunting, content providers will often go with a relatively quick-and-easy solution without addressing scalability or future needs. Of course, nobody can predict the future, but a good business analyst will look not only at current business requirements, but also ask questions about what might be coming down the pike in terms of user needs and requirements.

Here’s an example: Several organizations interviewed indicated that they utilize separate CMSs for each publication. That is, every major reference work (MRW) or journal has its own CMS rather than a centralized CMS, or an ECMS, to manage the organization’s content. This means editors and staff have to be trained on each system and, in some cases, they can number well above ten. The organizations know they need to move away from that structure, but have not yet begun making that change, even though it is hindering productivity, because they’re deciding how to execute on that plan.

Another example: Due to budget cuts and a change in strategic focus, one organization hadn’t been keeping its technology up to date and there were serious glitches that had to be addressed. The organization updated its technology to provide users with a far better user experience, thereby creating expectations for continued smooth operations and few-to-no glitches. At this point, if the organization does not keep up with updates and maintenance, supporting the current customer experience, there will come a time when it cannot keep up and will have to pretty much start at square one to meet user needs.

Copyright Clearance Center and the Book Industry Study Group (BISG) recently held a workshop for publishers exploring current technological challenges, which validated the above mentioned research. Their findings were revelatory:

• Many participants prioritized internal availability and searchability of data. That is, finding and analyzing data, as well as combining data from disparate systems to create a performance overview – something most publishers struggle with. Solving the problem is hindered in large part by a lack of internal resources, with strategic planning taking a back seat to “getting books on shelves,” but also by a lack of metadata discipline. A lack or inconsistency in metadata can mean the difference between adding and losing sales, as tagged content alerts publishers to potential sales opportunities.
• The lack of a centralized and/or digitized system to manage content was equally important, as published content is often recreated because it is either forgotten or can’t be found.
• Understanding user behaviors was cited as a major issue – even collecting user data on proprietary platforms can be a challenge, and with confounding factors like open access it becomes even more difficult.
• Also mentioned was the human challenge – most people are averse to change and when time/bandwidth is already at a premium, it is difficult to get staff to agree to learn new systems, workflows, and processes. This mindset severely hinders moving forward with technology upgrades.

 Publishers are now thinking about, and planning for, the needs of internal staff, authors, researchers, and institutions at each stage of the workflow. They are addressing them with, in many cases, decades-old bespoke technology, in some cases cobbled together with off-the-shelf modules. The modules do some of the work, but don’t always integrate well with legacy systems, so it’s still a mediocre patchwork in terms of functionality, reporting, and data collection and analytics.

Technology providers face many of the same challenges of figuring out what’s next, what users want now, and what they will want in the future. Some stand by their tried-and-true modular platforms with somewhat sophisticated functionality that can be customized to fit content provider specifications, but the ones who seem to be gaining the most traction, like 67Bricks and LibLynx, are those building on basic functionality to create bespoke solutions. Helping publishers formulate a technology roadmap and giving customers what they want is key, and what some providers, like 67Bricks and Copyright Clearance Center’s Professional Services (formerly Ixxus), provide.

It’s a difficult time for those facing important technology choices and implementations. It requires a deep knowledge of the scholarly communications industry as well as leading technologies in the marketplace that may or may not cater specifically to scholarly publishing. It also requires focus, sizable budgets, and a vision of the future. In a nutshell: Time, money, and vision.