Tuesday, August 18, 2009

Constructing Web Interfaces on the Fly

Web applications are dynamic- many requiring unique content and interfaces for each user. There are a myriad of ways to take user content out of a database and present it in a browser. This article focuses on the different ways it can be done in the Microsoft world (specifically in ASP.NET), why we chose XSLT for our own product, and finally a look at our technical implementation.

Planning for Dynamic Content

Whether you are in the process of building a new business web application or planning to build one, you most likely need to address how your web application will handle displaying different content for different users.

Considerations

When considering the different options for displaying dynamic content you generally need to take into account the following aspects:

  • Usability
    Usability might be one of the most critical aspects in the success (or lack thereof) of your application.
  • Development Time
    This includes the total amount of development time involved in satisfying your current application requirements.
  • Flexibility
    Regardless of how comprehensive your current requirements are, applications tend to evolve over time. It’s important to evaluate the development effort and skill sets required to accommodate changes.
  • Support, Maintenance & Ongoing Enhancements
    Commonly ignored by many when planning new development projects, it is generally responsible for a good chunk of the total cost of an application over the span of its life. This includes bug fixes, client customizations, minor application enhancements and of course, QA and testing.

The Options in ASP.NET

Generally web applications that use ASP.NET have two main options for displaying dynamic content:

  • Server Controls
    • With Binding – retrieving the relevant data and binding it to the appropriate ASP.Net server controls on a web form
    • In Code – populating the appropriate ASP.Net server controls in code
  • HTML
    • In Code – constructing the HTML to display in code based on the information retrieved from the database
    • With XSLT – retrieving the database information in XML format and then transforming it into HTML with XSLT

A third option is Silverlight, a new technology introduced as an option for web applications by Microsoft about a year ago. Silverlight provides a very rich GUI with the power of the .Net platform (a subset, actually) and tools that make web application development similar to the XAML interfaces found in the latest Windows application development. Silverlight is outside the scope of this particular article though.

Comparing the Different Options

Before you decide on the best route to take to display dynamic content you need to evaluate the impact of each approach on your application. You might find the table below helpful when making this decision.

Approach Pros & Cons When to Use
Server Controls
With Binding The quickest approach in terms of development time but also the least flexible. Best suited for applications with low data complexity and that are not expected to change often.
In Code Hooking up the controls in code takes longer in terms of development but is much more flexible than binding. This approach also requires a higher level of support and maintenance. Best suited for applications with high data complexity and that are not expected to change often.
HTML
In Code Constructing HTML in code gives a good deal of flexibility over the markup that’s created but is time-consuming and extraordinarily brittle. This approach requires extensive testing and support. I wouldn’t recommend this to be used as a general strategy for web applications but at times this might be used for particular sections depending on a unique set of requirements.
Using XSLT Offers high flexibility and the least amount of ongoing maintenance and testing. The development time might be longer depending on your team’s skill set. Best suited for applications with high data complexity and ones that are expected to change significantly and frequently. In addition, this allows for a lot of room for creativity as far as interface design goes.
Silverlight Provides the richest GUI possibilities and the complete separation of interface and code. The time to develop, test and support are currently higher than other options. Best suited for applications that require very rich user interfaces, and for development teams that primarily focus on Windows development.

Real-World Use Case: Scopings

When we started evaluating the requirements for Scopings, our homegrown recruiting platform, we needed a way to present complex content with a unique look and feel, a high level of usability, an infrastructure that can easily adapt to frequent and substantial changes, and built-in capabilities for globalization.

Very quickly we realized that although we can use ASP.Net server controls to build the first revision of Scopings in a relatively short period of time, this wouldn’t adapt well to the frequent changes we expected to be an inevitable part of the product lifecycle. This option would substantially increase our total cost of ownership.

After much analysis and many discussions, it became clear to us that designing our own infrastructure for constructing HTML using XML and XSLT would satisfy all of our requirements. We sat down and started to design an infrastructure that has ultimately been used for more than 80% of the functionality on Scopings.

Constructing the HTML

The idea behind the Scopings infrastructure was to allow us to make substantial changes to our user interface, while eliminating the need for any code changes and development staff involvement, and substantially decreasing the amount of QA required following any changes to the user interface.

To accommodate these requirements the Scopings infrastructure was built to be completely indifferent to the information retrieved from our database and to the way the information is ultimately displayed.

To achieve this we designed the infrastructure as follows:

  • A separate stored procedure was built against each web content page to be displayed, and was designed to only return XML back to the calling code.
  • Upon loading a web page, the code behind would load the XML for the current user. The XSLT would then transform the XML to an HTML interface for the user.
  • The web page would be responsible for all functionality, both on the server in the code-behind or on the client using Javascript.
  • All styling was handled with CSS.

Given the above mechanism, any change to the way we display data, or to the content of the page would only involve the following:

  • If any additional data is needed we would modify the stored procedure to include the new data required. If no additional data is needed, no change to the stored procedure will be made.
  • Modify the XSLT to include any new data to be displayed, and any changes to the display.
  • Modify the corresponding CSS files to make any changes required for styling.
  • Test display changes and specific page functionality if any.

This allows for content and interface changes that require little to no development efforts and can be done very quickly with minimal testing.

Constructing Web Interfaces with XSLT

In this section I’m going to provide the technical details needed to build a simple infrastructure for retrieving dynamic content from a SQL Server database and displaying it on a web page using XSLT.

As discussed above, the process of displaying dynamic content includes the following steps:

  • Extracting Database Content – building a stored procedure to output all web page content as XML.
  • Retrieving the XML from the Database – building a function to call the stored procedure and retrieve its data.
  • Transforming XML into HTML using XSLT – developing the XSLT required to transform the XML retrieved from the database into HTML.
  • Displaying Content on the Web Page – configuring the web page to retrieve the XML, convert it to HTML and finally display the interface to the user.

Step 1 - Extracting Database Content

The most flexible and powerful way to extract database content is simply to create stored procedures that output XML. This way all the data needed to for the interface can be easily extracted with one database call. A future change to the content only requires slight adjustments of the appropriate stored procedure without any changes to the calling code.

The following is a sample stored procedure that builds an XML document with a specific user’s first and last name, and a list of tasks this particular user needs to accomplish:

CREATE procedure [dbo].[Get_User_Profile_XML]
@User_ID varchar(10)
AS
SET NOCOUNT ON;
SELECT Users.[User_ID] AS [@id],
Users.First_Name AS [@first],
Users.Last_Name AS [@last],

--all user tasks
(
SELECT Tasks_ID AS [@id],
Task_Description AS [@description]
FROM dbo.Users_Tasks
WHERE Users_Tasks.[User_ID] = Users.[User_ID]
FOR XML PATH('task'),TYPE, ROOT('tasks')
)
FROM dbo.Users
WHERE Users.[User_ID] = @User_ID
FOR XML PATH('user'), TYPE;
Note: This was written against SQL Server 2005 and can be easily adjusted to use FOR XML EXPLICIT for use with SQL Server 2000.

The output XML of the above stored procedure might look as follows:

<user id="U12" first="John" last="Smith">
<tasks>
<task id="T34" description="Annual review"/>
<task id="T56" description="File my taxes"/>
tasks>
user>

Step 2 – Retrieving the XML Content from the Database

After building the appropriate stored procedure, we need to develop the code that can call this procedure on runtime and extract the appropriate data for the current user.

As increasing flexibility and lowering long-term maintenance costs are our primary goals, it is often best to keep the code needed to retrieve that database information almost entirely indifferent of the content it gets back. Extracting XML directly from the stored procedure gives us just that. We can keep changing the XML structure and content without requiring any changes to the calling code.

The following function might be used to extract the XML data from the stored procedure above and send the XML (as a string) back to the client:

private string GetUserProfileXML(string userId)
{
const string MY_CONNECTION_STRING = "YOUR CONNECTION STRING";
const string PROCEDURE = "dbo.Get_User_Profile_XML";


using (SqlConnection connection = new SqlConnection(MY_CONNECTION_STRING))
using (SqlCommand command = new SqlCommand())
{
connection.Open();

command.Connection = connection;
command.CommandText = PROCEDURE;
command.CommandType = CommandType.StoredProcedure;

SqlParameter param = new SqlParameter("@User_ID", SqlDbType.VarChar, 10);
param.Value = new SqlString(userId);
command.Parameters.Add(param);

using (XmlReader reader = command.ExecuteXmlReader())
{
reader.MoveToContent();
string resultXml = reader.ReadOuterXml();
return resultXml;
}
}

}

Note: This opens a connection to SQL server, configures the command object and its @ID parameter, and reads the XML as a string from the XmlReader.

Step 3 –Transforming XML into HTML using XSLT

Our next task is to transform the XML into HTML that can be displayed to the current user.

XSLT can be quite handy and an ideal solution when it comes to transforming XML into HTML. XSLT offers enormous flexibility, as it can be quickly adjusted to handle any changes to the XML content or changes to new requirements.

XSLT files can be easily included as part of any web application, and although this is beyond the scope of this article, development teams can choose to slightly adjust this approach and globalize their web applications by developing different XSLT files per language supported.

The following XSLT might be used to process the XML data above:

xml version="1.0" encoding="UTF-8" ?>
<xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform">

<xsl:output method="html"/>

<xsl:template match="user">
<div class="userInfo">
<xsl:value-of select="concat('Tasks for ', @first,' ',@last,':')"/>
div>

<div class="userTasks">
<xsl:for-each select="tasks/task">
<div class="userTask">
<a>
<xsl:attribute name="href">
<xsl:value-of select="concat('taskinfo.aspx?id=',@id)"/>
xsl:attribute>

<xsl:value-of select="@description"/>
a>
div>
xsl:for-each>
div>
xsl:template>
xsl:stylesheet>
Note: This outputs the heading for the page (first and last name) and then outputs every associated user task.

The HTML result (without any applied CSS) might look like the following:

Tasks for John Smith:
Annual review
File my taxes

Step 4 – Displaying Content on the Web Page

The final step is to create the web page needed to process and display the appropriate user content. As described at the beginning of the article, we need to retrieve the XML from the database by calling the "GetUserProfileXML" function we developed earlier, transform the XML into HTML using the XSLT file above, and then finally to display the HTML to the user.

We first create an ASPX page with a server side DIV to ultimately contain the HTML. The page might look like the following:

<%@ Page Language="C#" AutoEventWireup="true"  CodeFile="userprofile.aspx.cs" Inherits="UserProfile" %>

DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">

<html xmlns="http://www.w3.org/1999/xhtml" >
<head runat="server">
<title>My User Profile Pagetitle>
head>
<body>
<form id="form1" runat="server">
<div id="_divData" runat="server">
div>
form>
body>
html>
Note: The DIV is set to run on the server so the HTML can be assigned on the server.

Next, we need to develop the function that takes XML and transforms it into HTML using an XSLT file.

A function that accepts an XSLT file name and the XML data, and performs the transformation might look like the following:

private string GetPageHTML(string xsltFileName,string xmlData)
{
string fullXsltFilePath = Server.MapPath("~/" + xsltFileName);
using (XmlReader dataReader = LoadXMLToReader(xmlData))
{
XslCompiledTransform xslTrans = new XslCompiledTransform();
xslTrans.Load(fullXsltFilePath);
using (MemoryStream outputStream = new MemoryStream())
{
xslTrans.Transform(dataReader, null, outputStream);
outputStream.Position = 0;
using (StreamReader sr = new StreamReader(outputStream, Encoding.UTF8))
{
string resultHtml = sr.ReadToEnd();
return resultHtml;
}
}
}
}
Note: This function gets the full path to the XSLT file, loads the XML data into an XmlReader, and then uses XslCompiledTransform object to transform the XML into HTML and return the HTML as string to the calling code. Also, XslCompiledTransform is thread-safe object, so I would highly recommend caching it to further increase website performance.

You can use the following helper utility to load an XML string into an XmlReader object:

private XmlReader LoadXMLToReader(string inputXML)
{
byte[] xmlData = Encoding.UTF8.GetBytes(inputXML);
MemoryStream xmlStream = new MemoryStream(xmlData);
xmlStream.Position = 0;

XmlReader reader = XmlReader.Create(xmlStream);
reader.Read();
return reader;
}

Finally, to wrap everything up, here is what the Page_Load function on the web page might look like:

protected void Page_Load(object sender, EventArgs e)
{
const string XSLT_FILE_NAME = "UserProfile.xslt";

//gets the current user id
string userId = "U12";

//loads the xml data from the database
string xmlData = GetUserProfileXML(userId);

//transform the XML into HTML
string html = GetPageHTML(XSLT_FILE_NAME, xmlData);

//shows the html to the user
_divData.InnerHtml = html;
}
Note: The User ID is hard-coded but it should be retrieved dynamically based on the current logged-in user.

Conclusion

Dynamic content for web applications can be produced in several ways but if you are looking to build a web application with great flexibility, a high level of usability, and low total cost of ownership- using XSLT to transform XML into an HTML interface might be the right approach.

No comments:

Post a Comment