HTTP Request processing in details

The main role of the server-side code of a Morfik XApp is to receive and respond to Http requests. These Http requests usually come from the users of your XApp via a web browser, although they can come from other programs. When an XApp receives an Http request, it may need to perform certain actions with the data received (e.g. save a record to the database), or return certain data in the Http response (e.g. the html code of a form).


An Overview of HTTP Request Processing

The Http request processing operation is the “be all and end all” of all activities that take place in the server side within a Morfik application. The diagram in Figure 1 shows the key components involved in this operation.


HttpRequestProcessing.jpg
Figure 1 - Http Request Processing


Upon arrival of a request, the Web server creates a new execution thread and invokes the main entry point inside the Morfik application framework. The application's main object (a descendant of TXApp, eg. Project1XApp) receives the call and immediately creates a pair of Request and Response objects. By parsing the URL the application's main object determines the class of the HTTP server that is to serve the request and compose the response.

Before creating an instance of the HTTP server, all HTTP Filters that are installed into the framework are executed. The role of HTTP Filters are to alter or block an incoming request before the request is processed. Once a rquest gets through HTTP filters, an instance of the HTTP server is created and its execute method is invoked.

The execute method performs the main task in http request processing operation. Depending on the type of the HTTP server, the execute method performs tasks that vary from generating HTML and PDF to executing the code within a WebMethod.

Note: One important thing to bear in mind is that there could be multiple request processing operations in progress at any point in time within the server side of the framework. Each one of these operations run on its own execution thread and as such the execution of the server side code is highly concurrent. This includes all the application code written in the server side modules. Thread safety is an important issue on the server side and care must be taken in order to ensure threads are properly synchronized when accessing shared data. A typical example of this is when writing into an external file. The code that writes into the file must be written in such a way that only allows one thread of execution to run through it at any given point in time.

HTTP Request

An Http request is sent to a specific server, identified either by a host name (e.g. www.morfik.com) or by an IP address (e.g. 67.192.129.155). It includes the following:


Http verb (e.g. POST), followed by the name of the resource being requested;
Http headers;
optional message body.


The Morfik framework provides the THttpRequest class for working with the Http request. This can be accessed via the Request member of the THttpServer class. For example, the following code shows how one could log the IP address from which a webmethod call originated:

FX Code

Procedure WebMethod1.Execute;
Begin
    WriteToXAppLog('RemoteAddress=' + SoapServer.Request.RemoteAddress);
End;


Here is an example of an Http request:

POST /Index.htm?sbd=2&lng=&thm=ProjectTheme&tzo=-660&ins=Index&xid=%7B7166F464-9484-4DE3-8F44-
                   F00CF7C9C74B%7D&sid=&cls=form&ViewMode=vmSingle&PageName=Home HTTP/1.1
Accept: */* 
Accept-Language: de-at 
Referer: http://www.morfik.com/
Accept-Encoding: gzip, deflate
User-Agent: Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.0; Trident/4.0; GTB6.3; SLCC1; 
                         .NET CLR 2.0.50727; Media Center PC    5.0; .NET CLR 3.5.30729; .NET CLR 3.0.30729)
Host: www.morfik.com 
Content-Length: 0 
Connection: Keep-Alive
Pragma: no-cache
Cookie: __utma=57227574.1756966090.1260398203.1260491327.1260491779.5;
          __utmb=57227574;__utmz=57227574.1260398203.1.1.utmccn=(direct)|
            utmcsr=(direct)|utmcmd=(none); __utmc=57227574 

[message body]

HTTP Response

In response to an Http request, the server returns an Http response. This includes the following:


Http status code;
Http headers;
optional message body


The Morfik framework provides the THttpResponse class for working with the Http response. This can be accessed via the Response member of the THttpServer class. For example, the following code shows how one could force a form's html code to be returned in uncompressed format:


FX Code

Procedure Index.WebFormAfterExecute(Sender: TWebDocument);
Var
    HttpResponse : THttpResponse;
Begin
    HttpResponse := HttpServer.Response;
    HttpResponse.BlockCompression := True;
End;


Example of an Http response:

HTTP/1.1 200 OK 
Cache-Control: no-store
Allow: Post
Content-Type: text/html; charset=utf-8
Content-Encoding: deflate
Last-Modified: Fri, 11 Dec 2009 00:51:37 GMT
Server: Microsoft-IIS/7.0
Content: 
Date: Fri, 11 Dec 2009 00:51:37 GMT
Content-Length: 2807

[message body]

HTTP Servers

Pivotal to HTTP request processing are classes that descend from THttpServer. The THttpServer class is an abstract class that represents the processing of an Http request in the server-side code of an XApp. THttpServer has two memebers, Request and Response that provide access to the underlying Http request and response details. The key method of THttpServer class is the Execute method. The Execute method is overridden in descendant classes to perform the required handling of the request data, and to compose the response.

These classes provide the high level context for processing incoming requests. The application high level objects (Forms, Reprt, WebMethods, etc) have a member field that provides access to the underlying Http Server object as shown in the code below:


FX Code

Procedure About.TextLabel3BeforePrint(Sender: TWebControl; Canvas: TWebCanvas; Var Print: Boolean);
Begin
    TextLabel3.Caption := HTTPServer.Request.SecurityID;
End;


Note: In the case of WebMethods the corresponding field that provides access to the underlying HTTP server object is is SoapServer.


The Morfik framework provides the following THttpServer descendant classes for handling different kinds of Http requests.


HTTP Server Description
TPageServer Serves up the HTML code for pages.
TFormServer Serves up the HTML code for forms. Generation of the HTML code is handled by the TFormGenerator class.
TSoapServer Is responsible for receiving and returning XML code for webservice calls.
TSoapProxyServer Allows the server side of your XApp to be used as a proxy for other servers. This is used when making calls from the browser-side code of your XApp to a webservice on another server.
TReportServer Serves up the PDF code for reports. Generation of the PDF code is handled by the TReportGenerator class.
TBlobServer Serves up databound images and other binary content stored in the database.S
TResourceServer Serves up static images and other binary content included in the project resources.
TModuleServer Serves up the javascript code that Morfik generates for the project.
TFileUploadServer Handle files that users upload to the server.
THTTPFileServer Allows users to download files from the filesystem of the web server.
TDataFormCSSServer Servers up the CSS code associated with dynamic data forms.
TRSSServer Serves up XML code of RSS feeds.
TDebugConsoleServer Prints the Http request data received to the Output panel of the IDE. This can be useful for debugging purposes.


The built in Http servers are involved with key operations inside a Morfik application. They rely on compiler to generate application-specific code for objects that they manipulate. Their behavior is generally fixed, although can be overridden through a descendant class and subsequent registration of the server class as a custom server.

Custom HTTP Servers

Custom Http servers can be used to extend the capabilities of the framework beyond what is provided by the default built-in Http servers. Custom http servers are descendants of THTTPServer and can be defined inside any of the application server side modules.

A custom Http server must be made known to the framework so it can be called upon to perform its function. There are two ways to make a custom Http server known to the framework.


1- Registering a custom Http Server to serve a specific type of Http requests

The following function in SystemServer module registers a custom Http server class with the framework:

Procedure RegisterCustomHttpServer(Const RequestCls : String; ServerClass : THttpServerClass);

The first parameter specifies the type of Http request for which this server is to be invoked.

The following sample code shows a custom Http server that returns server's local time:

FX Code

Unit Module1;
 
Interface
 
Uses
   SystemServer,
   SystemUtilities;
 
Type
TPingServer = Class(THttpServer)
   Procedure Execute; Override;
End;
 
Implementation
 
Procedure TPingServer.Execute;
Begin
    Response.WritelnString(Now().toLongDateString);
End;
 
Initialization
   RegisterCustomHttpServer('ping', TPingServer);
End.


2- Overriding the GetHttpServer method of TXApp class

By overriding the TXApp.GetHttpServer method one can have a greater level of control over the way request checking is performed. If the request is checked and found to be targeting the custom Http server, an instance of the custom server is created and returned as the result of the TXApp.GetHttpServer method. The following sample code shows a custom Http server that returns server's local time:


FX Code

Unit Project1XApp;
 
Interface
 
Type
Project1XApp= Class(TXApp)
  Published
    { Events }
  Private
    { Private declarations }
  Public
    { Public declarations }
    Function GetHttpServer(Request : THttpRequest; Response : THttpResponse): THttpServer; Override;
End;
 
Implementation
 
Type
TPingServer = Class(THttpServer)
   Procedure Execute; Override;
End;
 
Procedure TPingServer.Execute;
Begin
    Response.WritelnString(Now().toLongDateString);
End;
 
Function Project1XApp.GetHttpServer(Request : THttpRequest; Response : THttpResponse): THttpServer;
Begin
    If Request.GetURLParameterValue('cls').Equals('ping') Then
        Result := TPingServer.Create(Request, Response)
    Else
        Result := Inherited GetHttpServer(Request,Response);
End;
 
End.

HTTP Filters

Sometimes you may wish to block specific Http requests received by your XApp. For example, you may wish only to accept requests coming from certain IP addresses, or to limit the size of uploaded files. Such tasks may be accomplished by means of Http filters.

An Http filter consists of a function, which must conform to the following signature:

THTTPFilterProc = Function (HTTPServer : THTTPServer; UserData : Pointer) : Boolean;


The Http filter function should return false and/or raise an exception if the request is to be blocked; otherwise it should return true. It needs to be registered by calling the following procedure, declared in SystemServer.mmd:

Procedure RegisterHttpFilter(FilterProc : THTTPFilterProc; UserData : Pointer);


where FilterProc is a pointer to the Http filter function. UserData is a pointer to user-defined data (may be left as nil).

Example 1. By default, the Morfik framework rejects all Http requests larger than a certain limit. The limit is 5 megabytes for file upload requests and 1 megabyte for all other requests. These limits are enforced by means of the following Http filter, declared in SystemServer.mmd:

FX Code

Function CheckAllowPost(HTTPServer : THTTPServer; UserData : Pointer) : Boolean;
Const
    HTTP_ERROR_413 = 'Http Error 413 - Request too large';
Begin
    Result := True;
    If StringsEqual(HTTPServer.Request.RequestMethod,'POST') Then
       If Not Catalog.postLimits.canAccept(HTTPServer.Request.GetURLParameterValue('cls'),
                                           StrToIntDef(HTTPServer.Request.ContentLength, -1)) Then
    Begin
        StrCopy(HTTPServer.Request.ServerImplementation.LogData, HTTP_ERROR_413);
        Raise Exception.Create(HTTP_ERROR_413);
    End;
End;
 
...
 
 RegisterHttpFilter(CheckAllowPost, Nil);


Example 2. We have a top-secret report, that should not be accessible to anybody outside the organisation. As a security measure, we add an Http filter to our XApp, that blocks any request for this report that does not come from a whitelist of known IP addresses.

We register our Http filter along with the list of IP addresses as follows:

FX Code

Procedure Project1XApp.XAppStart(Sender: TObject);
Var
    AllowedIPAddresses : TStringList;
Begin
    AllowedIPAddresses := TStringList.Create;
    AllowedIPAddresses.Sorted := True;
    AllowedIPAddresses.Add('67.192.129.155');
    AllowedIPAddresses.Add('67.15.211.4');
    RegisterHttpFilter(HttpFilter2, AllowedIPAddresses);
End;


The Http filter function looks like this:

FX Code

Function HttpFilter2(HTTPServer : THTTPServer; UserData : Pointer) : Boolean;
Var
    Temp : Integer;
Begin
    If HttpServer.Request.URLDocumentName.Equals('Top Secret Report') Then
        Result := TStringList(UserData).Find(HttpServer.Request.RemoteAddress, Temp)
    Else
        Result := True;
End;


This blocks any request for „Top Secret Report“ coming from a non-whitelisted IP address, but allows all requests that relate to other documents.

Http Request Headers

User agents (web browsers and other Http-enabled programs) include headers with the Http requests they send. Let's take another look at our sample Http request:

POST /Index.htm?sbd=2&lng=&thm=ProjectTheme&tzo=-660&ins=Index&xid=%7B7166F464-9484-4DE3-8F44-
                        F00CF7C9C74B%7D&sid=&cls=form&ViewMode=vmSingle&PageName=Home HTTP/1.1
Accept: */*
Accept-Language: de-at
Referer: http://www.morfik.com/
Accept-Encoding: gzip, deflate
User-Agent: Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.0; Trident/4.0; GTB6.3; SLCC1; .NET CLR 2.0.50727; 
           Media Center PC 5.0; .NET CLR 3.5.30729; .NET CLR 3.0.30729)
Host: www.morfik.com
Content-Length: 0
Connection: Keep-Alive
Pragma: no-cache 
Cookie: __utma=57227574.1756966090.1260398203.1260491327.1260491779.5; __utmb=57227574; 
            __utmz=57227574.1260398203.1.1.utmccn=(direct)|utmcsr=(direct)|utmcmd=(none); __utmc=57227574

[message body]

The values of these headers can be accessed using the THttpRequest class.

The Accept header specifies the media types that the user agent will accept in the response. It can be accessed as follows:

THttpRequest.Accept : String;


The Host header specifies the host name of the server. It can be accessed as follows:

THttpRequest.Host : String;


The If-Modified-Since header allows the client to specify that it only wants a fresh copy of the resource if it has been modified since a given date. It can be accessed as follows:

THttpRequest.IfModifiedSince : String;


The Referer header allows the client to specify the resource from which the request address was obtained. It can be accessed as follows:

THttpRequest.Referer : String;


The Accept-Encoding header indicates which compression algorithms are acceptable in the response:

THttpRequest.AcceptEncoding : String;   // e.g. 'gzip,deflate'


The Remote-Addr and Remote-Host headers identify the IP address of the client machine:

THttpRequest.RemoteAddress : String;    // IP address of the client machine
THttpRequest.RemoteHost    : String;    // host name of the client machine (blank if none available)


The User-Agent header identifies the program (usually a web browser) making the Http request. The value of this header can be obtained as follows:

THttpRequest.UserAgent : String;


This returns a string such as the following:

Mozilla/5.0 (Windows; U; Windows NT 6.0; de; rv:1.9.1.5) Gecko/20091102 Firefox/3.5.5


Rather than working with this string directly, it is usually easier to use the TBrowser class instead. This class includes a number of fields that represent specific pieces of information about the user agent.


The Engine and EngineID fields indicate the browser vendor:


Browser Vendor TBrowser.EngineID TBrowser.Engine
Safari 'SF' 'SAFARI'
Opera 'OP' 'OPERA'
Internet Explorer 'MS' 'MSIE'
Konqueror 'KQ' 'KONQUEROR'
Mozilla Firefox 'GK' 'GECKO'
Indy library 'IN' 'INDY'


(The Indy sockets library is an open-source library that supports Http and many other protocols. It is used in Morfik when calling a webservice published by another XApp.)

The Platform field of the TBrowser class indicates the operating system in use on the client machine:

THttpRequest.Browser.Platform : String;   // possible values: 'WIN', 'MAC', 'LNX'


The fields Version and MinorVersion indicate which version of the software is being used:

THttpRequest.Browser.Version      : Integer;   // e.g. 2009
THttpRequest.Browser.MinorVersion : Integer;   // e.g. 11


Finally, the Identifier field presents all five fields in a single string value:

THttpRequest.Browser.Identifier : String;   // e.g. GECKO(GK) 2009.11 WIN

Http Request Data

An Http request may have data attached. This could be, for example, xml code for a webservice call, or the content of a file that the user wishes to upload.

The data of an Http request may be accessed by means of the Buffer and BufferSize properties:

THttpRequest.Buffer     : Pointer;
THttpRequest.BufferSize : Integer;


Here is an example that shows how to save the xml code of a web service call to a file:

FX Code

Uses
    SystemCatalog;
 
Procedure WebMethod1.Execute;
Var
    MemoryStream : TMemoryStream;
Begin
    MemoryStream := TMemoryStream.Create();
    Try
        MemoryStream.WriteBuffer(SoapServer.Request.Buffer^, SoapServer.Request.BufferSize - 1);
        MemoryStream.SaveToFile(Catalog.RootDirectory + '\xml.xml');
    Finally
        MemoryStream.Free;
    End;
End;


If the request data is known to be plain text, the following function may be used to obtain the request data as a string instead:

THttpRequest.GetAsText : String;


In some Http requests, the data contains multiple parts. Such Http requests have value 'multipart/form-data' in the Content-Type header. The sections are delimited by a special boundary string, whose value may be obtained from the following property:

THttpRequest.BoundaryString : String;

Http Response Headers

Http servers attach various headers to the Http response to provide information to the client. Let's take another look at our sample Http response:

HTTP/1.1 200 OK
Cache-Control: no-store
Allow: Post
Content-Type: text/html; charset=utf-8
Content-Encoding: deflate
Last-Modified: Fri, 11 Dec 2009 00:51:37 GMT
Server: Microsoft-IIS/7.0
Content: 
Date: Fri, 11 Dec 2009 00:51:37 GMT
Content-Length: 2807

[message body]


THttpResponse class in the Morfik framework provides various fields for setting these headers. Setting Http response headers is something that would most likely be done in the context of implementing a custom Http server.

The Http status code indicates whether the Http request was successful, and may provide further information about the status of the requested resource. The Http status code can be set using the following property:

THttpResponse.StatusCode : Integer;


Possible values include:


Error Code Description
100 Continue
101 Switching Protocols
200 OK
201 Created
202 Accepted
203 Non-Authoritative Information
204 No Content
205 Reset Content
206 Partial Content
300 Multiple Choices
301 Moved Permanently
302 Moved Temporarily
303 See Other
304 Not Modified
305 Use Proxy
400 Bad Remote Call
401 Unauthorized
402 Payment Required
403 Forbidden
404 Not Found
405 Method Not Allowed
406 None Acceptable
407 Proxy Authentication Required
408 Remote Call Timeout
409 Conflict
410 Gone
411 Length Required
412 Unless True
500 Internal Server Error
501 Not Implemented
502 Bad Gateway
503 Service Unavailable
504 Gateway Timeout


The Content-Encoding header specifies the encoding (compression method) used in the Http response data. It can be set using the following property:

THttpResponse.ContentEncoding : String;   // e.g. 'deflate'


The Content-Type header specifies the MIME type of the Http response data. It can be set using the following property:

THttpResponse.ContentType : String;   // e.g. 'text/xml'


The Content-Length header specifies the size of the Http response data in bytes. It can be set using the following property:

THttpResponse.ContentLength : Integer;


Finally, the value of any named Http response header can be set by use of the following function:

Procedure THttpResponse.AddHeader(Name,Value : String);

Http Response Data

An Http response may optionally include data, for example, the html code for a report, or a file that the user wishes to download.

The implementor of an Http server can use the following method to write data to the Http response:

Function THttpResponse.WriteBuffer(const Buf : Pointer; Count: Longint): Longint;


If the response is in plain text, the following methods may be used instead:

Procedure THttpResponse.WriteString  (const S : String);
Procedure THttpResponse.WritelnString(S : String);


The methods are also available in the THttpServer class:

Function  THttpServer.WriteBuffer(const Buf : Pointer; Count: Longint): Longint;
Procedure THttpServer.WriteString(S : String);
Procedure THttpServer.WritelnString(const S : String);


It is also possible to load response data from a file, using the following method:

Procedure THttpResponse.LoadFromFile(FileName : String);


Related Topics


Back to top