Sirius Software, Inc. 875 Mass. Ave. Cambridge, MA 02139 Tel: (617) 876-6677 Mainframe Software Upgrade: Evolution, Not Revolution The Neverending Question With the advent of client-server and Internet architectures, IS faces an urgent recurrence of an old question: should it move today's mainframe software - especially business-critical software - off the mainframe and onto the new architectures' typical platforms? With cost and speed-to-market pressures at an all-time high, a correct choice can pay major bottom-line dividends. And while leaving mainframe software "as is" is less risky, the new architectures claim scalability, cost, rapid-development, and flexibility advantages that could compensate in the long term. Aberdeen finds that, on the contrary, the technology pendulum is swinging towards upgrading mainframe applications in place: Both client-server and Internet architectures have under-appreciated costs and risks that make it difficult to migrate large-scale, complex, mission-critical applications to them; Mainframe-software add-on suppliers are now so agile that mainframe software has new abilities to keep up with and integrate the new technology, plus its old proven benefits and experience in mission-critical production systems - you can "have your cake and eat it too"; and New Year-2000 tools and IS Year-2000 efforts make it easier to upgrade mainframe software in place. This Profile describes the troubles that moving mainframe applications to client-server and Internet architectures can cause, and the new capabilities that allow IS to evolve mainframe software systematically rather than tossing it away or revising it drastically in a "software revolution". Executive Summary As Internet solutions pervade many enterprises, IS is revisiting the question of whether to move legacy applications away from the mainframe. Many of these applications are "business-critical" ones that must deliver high performance and scalability in a production environment, because the business depends on it. Aberdeen finds that the new technologies such as client-server and the Internet, instead of tilting the balance towards moving the mainframe application to a new platform, are allowing IS to pursue a software evolution strategy that emphasizes continually upgrading business-critical mainframe applications to the new technologies, while keeping them in place on the mainframe. The argument for moving performance-critical applications onto servers has been that they can take advantage of the raw performance of server hardware and integrate more effectively with the new server software and technologies. Moreover, Year-2000 concerns make some IS managers feel that some mainframe applications cannot be salvaged, and should be replaced with "newer" solutions that are Year-2000-safe from their inception. Aberdeen finds that, on the contrary, migrating or rewriting performance-critical applications is now far harder than upgrading them. The resulting server application often delivers less performance, scalability, and robustness than the old mainframe one. Year-2000 tools used for mainframe-application upgrade and new technology-integration software from suppliers such as Sirius Software are making mainframe-application upgrade more feasible. What is true for performance-critical applications is often also true for other types of mainframe applications, such as custom enterprise applications with complex code, applications that require frequent and rapid changes because of end-user and business needs, and customer-interface applications that Web-enable back-office systems - all systems where conversion can yield major competitive-advantage benefits. These systems are typically critical to the enterprise's bottom line, and must continue to operate through the conversion process. While scalability and robustness after conversion are of less concern for these types of applications, migration and rewrite are similarly risky and effortful, and the migration may create performance problems where none were before. Therefore, Aberdeen urges IS to reassess current efforts to rewrite or replace performance-critical and similar mainframe applications, and consider instead a "best of both worlds" upgrade - mainframe-application robustness plus rapid development and the ability to keep pace with the rapid changes in technology. The New Opportunities Of Software Evolution Today's enterprises typically consider four choices for integrating new technologies with their existing application suites: upgrade in place. The program and its data are kept on the mainframe, and the developer applies mainframe tools that integrate the new technologies with the mainframe application. migrate. The program's source or binary code is moved to a new platform with little or no change, and the developer applies tools on the new platform to add the new technologies; the data may remain on the mainframe or be migrated as well. rewrite. The program is first "reverse engineered", creating a design model of the application, and is then generated from the model on the new platform; again, the data may remain on the mainframe or be migrated to the new platform. replace. A new program (using new data) is written from scratch on the new platform with the same functions as the mainframe one. In the past, enterprises tended either to replace mainframe programs or simply leave them "as is", finding any other options too difficult. Today, however, software evolution - defined by Aberdeen as upgrading, migrating, or rewriting legacy applications to use today's more cost-effective technologies - is more attractive than ever before. Evolution is becoming more cost-effective: new field-proven tools and technologies allow IS to evolve an application more easily, while the relative costs of maintaining "untouched" software or creating all-new applications continue to increase. Aberdeen research shows that the keys to evolving applications effectively are: Better information about applications to allow them to be upgraded or migrated more easily; Better tools for upgrading or migrating today's legacy applications; and Development processes that give IS greater strategic flexibility in choosing whether to upgrade or build anew. The new tools support all three. They allow IS to gather information vital to future upgrades or migration: previously lost information on a program's structure and connections to other programs, identification of unnecessary or flawed code, and some idea about which legacy applications are in the worst shape, far more cost-effectively and comprehensively - giving IS deeper information about a broad range of business-critical legacy applications. In many cases these new tools are field-tested in high-stakes Year-2000 efforts, making them better tools for upgrade or migration than have existed before now. Project management, version control, planning and budgeting, dependency tracking, conversion, and testing tools integrate to form a process-management toolset that can be applied to any application upgrade, migration, or rewrite. The Costs Of Revolution And Status Quo The two obvious alternatives to evolving a legacy application portfolio are "leaving it alone" and replacement (writing completely new applications - the "revolutionary" approach). Both are increasingly costly ways of doing business. Aberdeen research shows that many IS organizations spend little time planning to evolve key applications - often because they see no low-cost way of doing so. The result is constantly increasing "inventory costs" of legacy applications: Maintenance costs of applications about which key information has been lost and whose software and hardware is no longer adequately supported inside the organization or by suppliers; Opportunity costs as maintenance spending crowds out new-application development spending; and Inefficiency costs as the failure to proactively upgrade causes crisis-mode costly legacy-application fixes and periodic directives to "move to a new platform" waste time and effort on flawed or failed major upgrade projects. These costs often feed on themselves in a "vicious circle": choosing to maintain instead of upgrading an application means that the application continues to age, not only increasing maintenance, opportunity, and inefficiency costs but also making the gap between the legacy application and today's technologies wider and therefore making upgrade more costly. IS' usual Year-2000 efforts, if anything, are adding to the problem. Typical fixes such as windowing create a "pivot point" to put an application's dates in a "safe" range; this pivot point must be revised either yearly or at some point in the near future, adding to future maintenance burdens. Year-2000 conversion costs are crowding out other costs, increasing future "inventory costs". But writing all-new applications is also increasingly costly. As the application portfolio expands, the number of new applications and "replacement candidates" is expanding faster than IS programmer productivity. As we argue below, writing long-term scalable client-server or Internet applications can be especially costly and risky. Conclusion: change is needed - but it must be minimal change. Evolving Mainframe Software: Upgrade, Migrate, Or Rewrite? Today's enterprises typically consider three choices for evolving mainframe applications to new technologies: (1) upgrade, (2) migrate, and (3) rewrite. Superficially, choice 2 - and sometimes choice 3 - seems preferable to choice 1. In the three-tier architecture now popular in large-scale client-server and Internet implementations, putting an application on the second-tier application server would seem logical. However, in software evolution, less change means less problems - and "upgrade in place" involves the least change of the three choices. Aberdeen finds that there are two added types of problems that enterprises encounter as they try to move key mainframe applications to new platforms, rather than upgrading in place: The usual risks and costs of moving a business-critical application from a mainframe to another platform; and For performance-critical applications, the problems of delivering performance, scalability, and robustness comparable to the mainframe on the new platform. Risks and costs of moving key mainframe applications to client-server or Internet server hardware are massive. Mainframe applications - especially those highly tuned for performance - are so customized for the mainframe that they cannot simply be copied from one machine to another; instead, those migrating the application must have a deep understanding of the application's code and its purpose. Even then, migraters must rewrite much of the code in the application to run and deliver optimum performance on a very different type of computer - a matter of months or years. New errors may creep in during the process, making the resultant application unusable. Moreover, in many cases, the enterprise's knowledge of the application, or the documentation, has been lost or is inadequate. Today's reverse engineering or impact analysis tools cannot fully understand the performance-tuning code critical to the application's performance. For one set of mainframe applications, migration or rewrite is even harder - if not impossible. Many mission-critical mainframe applications are written in a 4GL like CCA's Model 204 User Language, with their data stored in databases and shared among highly-integrated applications. Migration and reverse engineering tools are lacking for these application environments. Moreover, if the data for one application is moved to a new platform and RDBMS - perhaps to integrate with applications on the new platform - then the applications remaining on the mainframe will need modifications to access their data from the new RDBMS, or application code must be written to keep the two databases in sync. Thus, for these types of application, migration tends to be "all or nothing" - if one application migrates, all must migrate. Enterprises' Year-2000 efforts are a prime example of just how difficult it can be to adapt an application to a new platform. The Year 2000 conversion process requires much the same tasks as migration or rewriting, but on a much smaller scale. However, as noted in a recent Aberdeen Viewpoint, because the applications are business-critical, converters must go through an elaborate process of planning, impact analysis, conversion, testing, and reintegration into the enterprise's production environment. The testing phase alone can take over a year. IS should also note that maintaining the migrated application on a server can be a long-term problem. Server tools are better and more experienced at developing new applications than upgrading old ones. Moreover, typically, the application continues to access mainframe data. As new end-user needs drive further changes in the application, developers must upgrade the application and the mainframe data in sync - a complex process. The Perils Of Migrating/Rewriting To Client-Server Aberdeen finds that the second set of problems - performance and robustness once the application is moved - is equally important and often more difficult to overcome, on both client-server and Internet platforms. Client-server and Internet architectures each are appropriate in specific situations - for example, client-server is appropriate for evolving today's major RDBMS-based applications and the Internet is appropriate for developing new Web-site applications or integrating enterprise networks. However, both client-server and Internet architectures have significant barriers to scaling data-intensive applications, and the tools for upgrading these applications are less experienced and robust than those on the mainframe. As users gain experience with client-server architectures, they find four particular areas where it is difficult to match the scalability and robustness of a mainframe application on the new architecture: Lack of guidelines as to where to split the application between client and server, lack of tools to support server-side programming, and often lack of tool flexibility to re-split the application when user needs change. In effect, users are often locked into an inappropriate client-server split that does not scale well and may deliver less performance than a comparable host-based application. "Fat client" problems. Lack of server-side tools means that applications can take up megabytes of space on the client side. This, in turn, has limited the number of applications that the client-server application can support. An increasing client-server-application legacy-code problem. Because of the complexity of the code needed to take advantage of client-server networks - and because users may lack client-server maintenance/in-ventory tools and focus) - IS is often finding client-server applications more difficult to maintain or upgrade than mainframe applications. This, in turn, can make them less robust and more difficult to scale. Proprietary development tools. While some development tools do span multiple RDBMSs and provide access to key mainframe data, the ones used most frequently by client-server developments are products supplied by the major RDBMS vendors. These RDBMS-vendor tools typically focus on the vendor's own RDBMS, and therefore make it more difficult than on the mainframe to access multiple relational databases or host data. This, in turn, makes it more difficult to scale many data-intensive applications. Enterprises have also often found that "fixes" to "client-server-enable" mainframe applications in place - e.g., "screen scrapers" to provide a remote procedure call mechanism - are relatively straightforw and require little additional code. The Perils Of Migrating/Rewriting To The Internet As noted in a companion Profile (The Case For Web-Enabling Mainframe Applications), achieving moved-application performance and robustness on Internet platforms can be as difficult as on client-server platforms, or more so. To put it bluntly, the Internet is not "ready for prime time". Developers must write Web applications using tools that have far too little experience in delivering performance in data-intensive situations or in adapting to the unique Internet architecture. Likewise, the middleware (such as TP monitors) that these developers must invoke is immature, not robust, or not scalable. In fact, it is not yet clear just what is the right way to scale a data-intensive application on the Internet: what is the right split between client applet and server, or between application servers. Aberdeen research shows that developers have two basic choices when creating or Web-enabling performance-critical, data-intensive Internet applications: Write on top of the CGI set of Web-server APIs; or Write on top of middleware that bypasses CGI, such as Oracle's Network Computing Architecture or Microsoft's Wolfpack. As a first-generation Web TP monitor, CGI uses much more CPU time than necessary. Neither it nor the Web server was designed to handle transactions, so each transaction must be chopped up into request-reply interactions with no knowledge of the transaction's "state" between the interactions - a major source of unnecessary overhead and developer effort. CGI's interface is likewise very low-level and has no knowledge of data - placing the burden of creating not only a data access mechanism but also much of the database management system software on the developer. CGI does not offer the key TP-monitor scalability technologies proven in over 25 years of mainframe and Unix applications (communications multiplexing and load balancing). Thus, using CGI to build scalable, robust applications is a bad idea. The evidence for this in real-world application development is already clear: ISVs developing Web solutions testify that CGI-based applications act as a "scalability bottleneck" on the server, slowing communication through the Web server to the back end - and especially for data-access operations. As a result, CGI-based applications - the majority of today's Internet applications - are not scaling rapidly enough in many cases to match today's end users' exceptionally fast-growing Web-data-access demands. CGI-based applications are also more difficult to build and maintain than most other applications. Key technologies for scaling Microsoft Internet environments are Microsoft's IIS (a CGI equivalent that is more lightweight and includes communications multiplexing), Wolfpack (a TP monitor based on Tandem's technology), and Application Server (providing basic support for load balancing across applications on a single server). While IIS has demonstrated cost-effective scalability in some cases, it clearly does not provide the developer support and load balancing scalability of a TP monitor. Wolfpack is not yet widely available and is in any case aimed at less complex applications than the typical performance-critical mainframe application. Application Server offers limited load balancing that must be programmed in by the developer and is not yet widely deployed. None of these technologies has been field-tested in scaling enterprise-scale data-intensive applications - and Microsoft is not focusing on delivering high-end scalability for Internet environments. Overall, Microsoft-related middleware products do not deliver robustness or scalability comparable with mainframe TP monitors, and because of the products' immaturity and Microsoft's focus elsewhere it is not likely that a fully integrated suite of mainframe-class products will be available in the next 1½-2 years. Non-Microsoft solutions also have scalability or robustness problems: Their TP monitor functionality has not yet been fully tested in large-scale enterprise Intranets and Web sites; Some of the technology needed to support location-independent operation in the distributed Internet environment - e.g., ORBs, repositories, and partitioning tools - is not yet integrated together, much less field-tested as a whole; and It is not yet clear what is the best way of partitioning applications across servers - or of partitioning applications between client or applet and server - in the new Internet architectures. The Migrate/Rewrite Bottom Line: Not What You Hope Aberdeen concludes that most if not all migrations or rewrites of mainframe business-critical applications will not deliver as hoped: Development will be harder and longer than expected. Initial success in small-scale implementations can hide the difficulties of implementing complex and data-intensive performance-critical applications. Reproducing the application on a new platform - if it is possible - requires an extensive process. Because the basic tools to support scalable, robust applications are not yet there or not integrated, the developers must either add those tools by hand - a long and tricky process - or have the application fail to scale in the field. The application will be less robust than it was on the mainframe. Errors and problems may creep in during migration or rewrite, and the middleware on the server supporting the application is not yet as robust as on the mainframe. The application will often perform less well - to the point of unuseability - than on the mainframe. Certainly, today's server hardware has impressive raw power and supports the latest software technologies. However, as we have seen, these technologies have not achieved the level of scalability that today's mainframe software delivers for performance-critical applications. Upgrading Mainframe Applications In Place: New Flexibility Aberdeen finds that upgrading mainframe applications in place is more feasible than ever before, for three reasons: Upgrade tools are better. A side-effect of enterprises' Year-2000 efforts is a thriving market in field-proven tools to upgrade mainframe applications. While, as noted above, any change to a complex or performance-critical mainframe application is not easy, upgrading that application is less of a change than migrating/replacing it, and one that many enterprises will already have done in Year-2000 conversion. Moreover, Year-2000 efforts yield application information that can make upgrade easier. Mainframe software suppliers are doing better at staying abreast of client-server and Internet technology. Networking and gateway products such as IBM's DDCS (now DB2 Connect) and mainframe TCP/IP give new client-server-support connectivity from servers and PC clients to mainframe applications. Solutions such as IBM's Net.data provide connectivity from mainframes to Internets, ensuring that mainframe applications can connect to the Internet and Intranets effectively. Solutions are arriving to allow enterprises to convert many business-critical mainframe applications effectively in place, by adding gateways to and from PC clients and RDBMS servers, or by Web-enabling or partitioning them. Sirius Software's Janus products provide low-overhead integration between Sybase's highly open Open Client/Open Server/Omni-Connect gateways and business-critical Model 204 applications. Janus Web Server allows users to Web-enable Model 204 applications rapidly, preserving their scalability and robustness. IBM's Net.data and DB2 Connect IBM's "information integration" solutions include Net.data and DB2 Connect, both of which are being folded into IBM's DB2 RDBMS. DB2 Connect (formerly DDCS) provides access from all supported client platforms to DB2. Net.data aims to allow businesses to deploy interactive Web-based applications that leverage existing application logic and data. Net.data DB2 WWW Connection enables developers to write HTML/SQL-based macros that handle Web-server data access. Thus, both Web pages and end users issuing queries can access or update information in backend databases "dynamically", i.e., immediately, without bringing down or bypassing the Web server. Developers can not only create macros from scratch, but also by cut-and-paste from Web authoring tools. Net.data delivers higher performance by allowing native access to DB2, Oracle, and Sybase databases, and native APIs rather than CGI. Net.data also includes ODBC-gateway and native flat-file support and supports DB2 Extenders for Internet-multimedia-data access. Net.data allows developers to place application logic on the client, Web server, or database server as appropriate. Net.data provides APIs and development tools for Java, C/C++, COBOL, and other server-side "backend" applications - thus, users can quickly redeploy existing logic into new Web-based applications. Net.data supports invocation of DB2 stored procedures, allowing higher performance for commonly-used data-management operations. Net.data allows Web-site developers to use "variable substitution": creation of variables that can be initialized by end users or backend programs (e.g., from a database) and changed by the administrator or backend programs while the Web server is running, allowing constant update of Web displays. Developers can also use these to keep track of an end user's Internet connections associated with accesses to a backend database - i.e., the "state" of a DB2 transaction between connections. Thus, a Web site can ensure that OLTP and decision-support transactions are carried out correctly. Sirius Software's Janus Products Computer Corporation of America's Model 204 has long been known as a "diamond in the rough" - an often-overlooked mainframe DBMS with innovative technologies that deliver exceptional scalability and rapid development of data-intensive applications. Thus, Model 204 User Language has been for the last 15 years a 4GL of choice among knowledgeable mainframe developers creating performance-critical applications. Sirius Software's Janus products tightly integrate support for industry-standard TCP/IP protocols with Model 204's development environment, allowing Model 204 databases and applications to become full participants in network-based application architectures such as client-server and the Internet. Janus TCP/IP Base provides a direct, high-performance connection between Model 204 and a TCP/IP-based network. Janus Network Security supports the Secure Sockets Layer (SSL) protocol. Janus OmniSQL Access Model operates in conjunction with the Sybase OmniConnect server to provide dynamic SQL access to data stored in Model 204 databases. Janus Open Server allows Model 204 applications to act as servers in an open client-server environment involving multiple databases from multiple suppliers. Janus Open Clients allows Model 204 applications to act as clients in an open client-server environment. Janus Web Server extends Model 204 to allow Web-browser access to Model 204 databases. Moreover, Janus Web Server includes a Model 204 User Language API, allowing Model 204 User Language developers to extend their applications to support Internet access simply and rapidly. Janus Web Server includes support for IBM's SSL security scheme, ensuring vitally-important key-corporate-database security. An upcoming extension of Janus Web Server will support a CORBA Object Request Broker (ORB), allowing creation of Internet applications with location-independent access to persistent database objects in Model 204 databases. Upgrading a Model 204 application with Janus Web Server is clearly preferable to migrating it or rewriting it on a Web server using CGI or some other mechanism. Model 204 User Language is far higher-level than CGI and includes far more knowledge and experience of data-intensive applications - resulting in far greater programmer productivity - and has demonstrated scalability and robustness far beyond CGI. As a 4GL, Model 204 User Language is a match for any Web-server development tool; Model-204-based mainframe applications are typically more robust than non-CGI environments; and by placing the Web-enabled mainframe application on the mainframe, the user can take some of the load off the Web server and improve the mainframe application's performance. Where Mainframe-Software Upgrade Tools Can Be Most Effective Aberdeen finds that mainframe-software upgrade tools can be most effective: As part of a Year 2000 effort. Just as Year-2000 is improving mainframe upgrade tools, so too can the new upgrade tools improve Year-2000 conversion. Their support for COBOL and business-critical mainframe data conversion allows more rapid mainframe-software upgrade, and collects key information for further upgrades in the future. As part of an overall application-evolution strategy. By making it easier to upgrade mainframe applications in place, these tools give IS greater ability to apply cost-effective upgrade rather than risky and costly rewrite or migration, and give IS much greater flexibility in choosing which applications to evolve first. Where powerful software and databases such as Model 204 are part of the enterprise architecture. By combining upgrade tools with other strong-technology mainframe products, IS can put mainframe software on a higher technology "growth path" than with only upgrade tools. Aberdeen Conclusions The advent of client-server and Internet architectures is good news for IS - but not in the way one might expect. Instead of making it easier to move mainframe software to new platforms and thereby keep pace with software-technology improvements, these architectures are by and large offering only costly and risky migration to potentially less robust and lower-performance versions of the application. However, new tools for keeping pace with client-server and Internet technologies and upgrading mainframe software in place give IS a new ability to pursue a software evolution strategy. IS can now upgrade mainframe software much more rapidly and cost-effectively, as well as setting the stage for further and easier upgrades as future technologies arrive. Thus, enterprises can "have their cake and eat it too": support clients and second-tier application servers or Web-enable in place with minimal impact on the enterprise's production environment. Therefore, Aberdeen recommends that senior IS managers, instead of continually revisiting the question of replacing, rewriting, or migrating mainframe applications, seize the opportunity to pursue a software evolution strategy that upgrades business-critical mainframe applications in place. Thus, IS can continually meet the new enterprise technology and competitive-advantage-application needs, with minimal negative impact on the systems running the business - a true "best of both worlds". AberdeenGroup, Inc. One Boston Place Boston, Massachusetts 02108 USA Tel: 617.723.7890 Fax: 617.723.7897 Contact Information: For further information on AberdeenGroup's products and services please contact us at info@aberdeen.com This Document is for Electronic Distribution Only -- REPRODUCTION PROHIBITED -- Copyright 1997 Aberdeen Group, Inc., Boston, Massachusetts The trademarks and registered trademarks of the corporations mentioned in this publication are the property of their respective holders. Unless otherwise noted, the entire contents of this publication are copyrighted by Aberdeen Group, Inc. and may not be reproduced, stored in a retrieval system, or transmitted in any form or by any means without prior written consent of the publisher.