Meta

Easysoa HTTP Mining Feature

30 May, 2012 (10:53) | Technical | By Jérémie Guillemotte

The goal of HTTP Mining is to have a system allowing to replay existing service exchanges in a parameterized manner, coupled with an assertion engine. It is built on previously developped features.

General architecture of Http Mining :

The HTTP Mining feature uses several EasySOA SCA components designed to work with FraSCAti :

  • Run manager : Designed to record a set of HTTP exchanges. For now, the run manager can be deployed in the HTTP proxy or in a servlet filter. A set of exchanges is a ‘run’.
  • Correlation engine : Used to find interesting fields in an exchange (mainly JSON / HTTP). It does so by correlating on value and path atomic input and output fields. These interesting fields are then used by the template engine.
  • Template engine : builds parameterized exchanges from recorded exchanges and interesting fields found by the correlation engine. Parameterized exchanges are stored as Velocity templates that are to be used by the replay engine.
  • Replay engine : This component can be used to replay one exchange or a complete run with or without customizations. The replay engine can be seen as a client, resending a recorded request to get a new response from the targeted service.
  • Assertion engine : Used to make assertions between a recorded exchange and a replayed exchange.
  • Simulation engine : This component can be used to simulate a parameterized exchange response: the simulation engine acts as a server, listening about a request and returning a customized reponse. This engine works with the correlation engine and the template engine to detect the most interesting fields to templatize.

Each of these engines are SCA components designed to run on FraSCAti.

How HTTP mining works ?

HTTP mining regroups the replay engine and the simulation engine.

In both cases, the first thing to have is at least one recorded HTTP exchange containing a request and a response. This exchange can be created by the HTTP discovery proxy or by the Nuxeo REST filter.

For the replay engine :

The replay needs a recorded exchange to work. It takes a request and resends it, then waits for a response from the original server. When the response is received, a checking mechanism is used to verify the similarity of the received response with the recorded response.

For the simulation engine :

The simulation engine waits for an incoming request. When a request is received, a matching mechanism is used to find a corresponding recorded exchange, then the a corresponding customized response is returned. The correlation engine is first called to find the most interesting input/output fields that can be parameterized (for example fields for ID’s). It works by correlating atomic input and output fields of JSON / HTTP exchanges. Note that it would work the same with XML / HTTP, but correlation is better done on type definitions / schemas when there are any. A list of those selected fields is established and passed to the template engine.

The template engine makes a copy of the exchange and replaces the selected field values with a Velocity expression. This expression will be replaced at runtime by default values from the original exchange or by custom user input values. The templates, one for the request and one for the response, are recorded in EasySOA.

Here is a sample of template field suggestion generated by the correlation engine. The suggested field is the ID which can be found in the query parameters :

{
    "templateFields":[
        {
            "correlationLevel":4,
            "defaultValue":"95379276",
            "fieldEquality":true,
            "fieldName":"id",
            "fieldType":"",
            "paramType":"IN_QUERY_PARAM",
            "pathParamPosition":0
        }
    ]
}

Here is a sample Velocity template generated by the template engine. The ID field value (line 44) is replaced by a Velocity expression. At the rendering, the expression will be replaced by a custom ID value or by the default value if no custom value is provided :

#macro ( renderReq $arg0 $arg1 $arg2 )
{
    "comment":"",
    "headers":{
        "headerList":[
            {
                "comment":"",
                "name":"Host",
                "value":"localhost:8088"
            },
            {
                "comment":"",
                "name":"Proxy-Connection",
                "value":"Keep-Alive"
            },
            {
                "comment":"",
                "name":"User-Agent",
                "value":"Apache-HttpClient/4.1.1 (java 1.5)"
            }
        ]
    },
    "messageContent":{
        "JSONContent":false,
        "XMLContent":false,
        "comment":"",
        "contentType":"Undefined",
        "encoding":"",
        "mimeType":"",
        "rawContent":"",
        "size":0
    },
    "method":"GET",
    "path":"/1/users/show/FR3Aquitaine.json",
    "port":8088,
    "postData":null,
    "protocol":"HTTP",
    "protocolVersion":"1.1",
    "queryString":{
        "queryParams":[
            {
                "comment":"",
                "name":"id",
                "value":"$arg2.get("id").get(0)"
            }
        ]
    },
    "requestTimeStamp":1338282406476,
    "server":"localhost"
}
#end

Next step :

Next step is to have a GUI to drive the simulation and the replay engine, integrated in FraSCAti Studio.

Comments

Pingback from EasySOA » EasySOA 0.4 Release
July 26, 2012 at 5:20 pm

[...] Proxy enriched with new HTTP Mining features, and new Service Event Subscriptions handler in proxy, whose configuration can be managed [...]

Write a comment





Couldn't connect to server: php_network_getaddresses: getaddrinfo failed: Name or service not known (0)