Forms, Workflow, and the Web
By Tom Spitzer
My company provides products and services to support the development of user interfaces for business applications for the Web. Our early efforts developing PC accounting software in the '80s were to emulate preprinted forms as the user interface for entering invoices, purchase orders, returned merchandise authorizations, and checks. Our software began its life as a character-mode application written in dBASE, so entry screens were functional but had nowhere near the detail of preprinted forms.
In the early '90s, we experimented with electronic forms tools, hooking them up to server-side databases that incorporated business logic in their stored procedures. These products, with names like PerformPro, FormEasy, and JetForm, had been introduced to enable institutions that relied heavily on forms processing -- insurance companies, banks, and government agencies -- to automate some of that processing. Vendors introduced these products to support legal and record-keeping requirements rather than to provide tools for developing general-purpose user interfaces. As a result, we never incorporated them into our product-development tool sets, opting instead for general-purpose tools like Visual Basic, PowerBuilder, and Visual FoxPro. Recently we recalled that experience as we struggled to buy or build an effective user interface for designing forms-oriented Web applications.
Then Came the Web
Ever since the emergence of the Web, with its simple
<FORM> tag supported by CGI programming, people have built relatively simple forms applications into Web pages. This facility created the first universally standard, albeit dumb, cross-platform user interface. With some reasonably simple scripting, validating data entry and presenting a series of forms based on user actions became a relatively straightforward process. There are several challenges associated with this technology, including session and application-state management. Supporting workflow requirements with HTML requires extensive CGI programming, which tends to be brittle and not very scalable.
The most important issues for us related to the user interface. The HTML approach yielded user interfaces that lacked the richness that our clients had come to expect. It also couldn't provide the level of detail and faithfulness between onscreen and printed representations. Existing HTML editors enable creation of rich informational content, but can't create sophisticated forms-based applications or define workflow in any meaningful way. Our company wanted better tools to create user interfaces for applications that involve a lot of data entry and data presentation than what was offered by low-priced tools. Ideally, such tools would let us define workflows, such as routings between users for further action and approvals. We thought that the forms products being delivered for use on the Web might provide us with the functionality we sought.
Contrasting Forms with Function
Before terminals and desktop computers became common, institutions used forms to record information. Some institutions still require people to fill out paper forms when seeking jobs, medical care, admission to educational institutions, and insurance coverage, among other things. In the early days of automation, completed forms were submitted to a data entry queue where the data they contained was rendered onto punch cards. Later, organizations used terminals to collect the data, while sophisticated user interfaces enabled literal transcription of forms to the screen, which reduced the amount of paper used at the beginning of the process. To this day, many organizations use these systems to print out copies of completed forms for storage. Creating, distributing, collecting, transcribing, and archiving forms is still a major institutional function, although today sophisticated computerized processes exist to support it. This is necessary because many forms provide evidence of legal relationships. Whether you fill out the form on paper or onscreen, it needs to be signed by the parties involved as evidence that they understand both the data entered on the form and the information context of the form. In industries like government and healthcare, regulations govern the structure and layout of the form regardless of the media used for its display. Filling out a form typically represents an event in a larger flow of work. Forms typically get routed through several stages of review and approval or denial before they're archived. During the '90s, business-process reengineering efforts eliminated some of the steps and some of the paper from these processes, but those in which the form serves a contractual function have not yet been able to leave the paper behind.
Forms and Workflow
Since many applications revolve around processing data collected in forms, it seemed logical to use the Web to build these applications and make them accessible to larger populations. In fact, a number of tools have become available recently that support the migration of form and workflow applications to the Web. These tools need to address the legal and workflow requirements that characterize traditional forms-based applications. In addition, they face the significant challenges presented by any Web application designed for large numbers of people.
Depending on the application, Web-based form tools need to fulfill a diverse set of requirements. In all cases, they need to support the development of user interfaces that are both usable and data intensive, and that enable deployment to Web browsers with a minimum of special demands on bandwidth and browser capabilities. For applications where the form represents a step in a larger workflow, the form tool needs to integrate seamlessly with a scalable workflow management system. In many cases, they need the ability to faithfully print and store forms and provide a way to efficiently and accurately retrieve a specific form from what could be a large repository. So they're often integrated with document management systems that collect a wide variety of forms and documents, index them, and maintain their integrity by controlling multiuser access and versioning.
Many organizations have realized this link between forms and workflow and are working on tools that leverage both technologies. Workflow automation systems provide tools for defining, executing, and monitoring processes in which multiple people must collaborate to meet a specific objective. They support process definition with a graphical modeling tool that lets someone lay out a workflow visually. People participating in a workflow interact directly with a workflow engine, a special-purpose application server that sits behind a Web server. The workflow engine generates administrative pages for monitoring all of the activity in the system as well as custom pages for each participant, displaying the work that's currently assigned to them.
When selecting tools for implementing workflow solutions, it's helpful to classify applications into one of the three categories that are regularly used by workflow practitioners. These include production, administrative, and collaborative workflows.
Production workflows provide a direct service to a customer or internal party. Examples include processing an insurance claim or loan application. In this category, forms are tightly integrated with workflow, participants tend to work full-time on tasks associated with the process, and the range of actions allowed for each participant is tightly constrained. As a result, workflow execution needs to prevent bottlenecks that can leave participants idle. Productivity improvement requires sophisticated rules-processing engines that take the place of human decision makers. To define production work processes, these applications typically require tools that support complex workflows as well as APIs for integrating with legacy applications with which they interact.
Administrative workflows are typically less complex. Examples include travel and purchase requisitions and expense reimbursements. Many people participate in administrative workflows, but they do so casually and occasionally. In many cases, the distributed forms and documents originate and reside outside of the workflow system. Organizations tend to have many more administrative workflows than production workflows, and management of these workflows is decentralized. As a result, ease of process definition is more important than the ability to model complex processes. Administrative workflows typically experience much lower throughput than production workflows.
The third type of workflow application supports collaboration among knowledge workers. Examples include engineers and architects working on blueprints for a building, or legal and accounting reviews of financial reports. Collaborative workflows are focused on supporting the production of a document or set of documents. Often they're connected to document-management systems. While collaborative workflows are designed to anticipate a standard flow of documents among a predefined set of participants, they need to enable flexible routing and ad hoc extensions, since they often require including additional people in the workflow under exception conditions.
Forms vs. <FORM>s
There are several situations for using forms-oriented tools for Web applications. The most obvious is when an institution is already using electronic forms, and wants to take advantage of the benefits of deployment over the Web. Conversely, many companies may wish to take advantage of an existing Web infrastructure to automate a process that exists in paper form that, in many cases, must ultimately be printed. Both reasons support the use of form development products instead of a vanilla HTML/CGI solution, as most will let you scan a paper form and convert the image to a digital representation. The need to print forms can drive this selection as well, since these products enable the output of paper documents that resemble the originals.
We also wanted to adopt a tool that would serve as a development environment for user-interface components that would ultimately be generated by a middle-tier server. For our application, a couple of the products we looked at (JetForm's FormFlow and UWI's InternetForms Commerce System, or ICS) would work fairly well. Both produce an XML representation of the form, but each uses a different XML-based description language. Since we drive our generator with our own XML-based page description language, we could translate the output of the forms tool from the vendor's syntax to our own. We haven't invested in developing the translation program since we don't think there is sufficient market demand. Neither have we been sufficiently impressed with the design capabilities of any of the forms tools to consider packaging them as part of our own solution.
We also had to consider developing at least rudimentary rules capabilities that would run on a middle-tier server as an alternative to buying a workflow system (see the box titled "
Workflow Roundup"). These systems include Web applications that serve users and administrators, so when somebody logs on, the system generates a page that presents the appropriate tasks. You could build applications to do this, but you'd have to include a substantial amount of intelligence. In addition, the available workflow products provide very advanced user interfaces for developing workflow maps and defining routing and processing rules that would be difficult to duplicate without a significant amount of effort. My approach as a solutions developer would be to use a workflow system that integrates with a forms tool, such as
Action Technologies Metro, which works with the UWI forms tool, or JetForm InTempo, which works with JetForm's own FormFlow system.
When considering a forms tool for a Web-based data collection, you need to determine whether a form is appropriate, or just adds overhead. It's also important to consider the security aspects of deploying a forms-management system, and to determine whether and how you can impose a security model on the form and its processing. This can be a challenge in the absence of widespread adoption of public key infrastructures. Since you'll typically deploy Web forms both within and outside of the enterprise, you can't rely on internal OS-based security models. As such, we recommend against using forms systems for data-collection applications and applications that function as "replacements" for client/server systems. We're more comfortable using server-based Web technologies such as ASP or JSP for such applications.
Finally, it's worth evaluating whether the infrastructure can really support a forms application, particularly one in which you expect large numbers of people to participate. Early forms tools relied on email transports for routing forms. Email offers many benefits since it's a narrowcast mechanism that offers the ability to tightly control routing. You may want to continue to use email-based forms systems (still offered by companies like JetForm and Shana) if you're developing an intranet-based application in which all of the participants have addresses on a common mail system. The Web infrastructure lets you extend participation to people outside your corporate mail server, but demands more bandwidth and imposes additional requirements. Typically, this means integrating the form processor with a workflow process that may be Web or email based. Since you can extend forms applications to customers, suppliers, applicants, and other newly accessible constituencies, you can build a much wider variety of forms-based applications. I was surprised to learn how much of the work being done with Web-based form tools is new work, as opposed to reengineering of existing forms applications.
In future articles, I'll delve into the technical aspects of some of the leading form and workflow products, and look into their support for the Web, particularly those that are using XML as a declarative language for defining forms, form routing, and workflow processes. I'm also anxious to see what the forms vendors are doing to support a development approach that addresses usability concerns. I'll be thinking about how to gauge the usability of such applications, and how to integrate them with related applications that are appearing on your Web. I'd welcome your input on the subject.
Tom is VP of product management for EC Wise, which helps companies get early user feedback on the features, functions, and usability of proposed Web-based products. You can reach him at email@example.com.