Manifest File Generator

There are many times when we encounter cases requiring us to write small programs which can be invoked using a bat file utility. How do we do that.. simple create an executable jar file and write a bat file to invoke th esame from command prompt. All that you need to do to make a jar file "Executable" is to create a manifest.emf which contains the class path and the location of main(). Here is a sample Manifest file:
Manifest-Version: 1.0
Class-Path: mgapi.jar rvconfig.jar THEMD.jar THEOE.jar THEOM.jar THETHE.jar THETransport.jar tibrvj.jar tibrvjsd.jar tibrvjweb.jar tibrvnative.jar tibrvnativesd.jar
Main-Class: com.ubsw.moap.publisher.MoapPublisher
If you have a look at the "Class-Path" attribute it contains list of all jar files referenced separated by a space " ". What happens when you have a project which references say 40 jar files. This line will become too long and you executable jar file will not work (it throws a stupid array out of bounds exception i guess !!!). So, what's the solution??? Well you need to create a bat file in the following format:
Manifest-Version: 1.0
Created-By: kumarprc
Signature-Version: v1.0.0
Class-Path: lib/activation.jar
__lib/ant.jar
__lib/axis-ant.jar
__lib/axis.jar ... and so on note that the trick lies in keeping two spaces " " before start of next line. and Finally ending with
__lib/xercesImpl-5.5.12.jar
__lib/xml-apis-5.5.12.jar
__lib/xsdlib.jar
Main-Class: com.ubs.sats.TestManager.TestRunner

To create this manifest file manually is too much pain and trusting eclipse to do it for you (packaging.. creating jar and manifest) generally results in "Class not found error" heartburn. So what i do is create a lib dir in my project and keep all my jar files there. Then i use the following program to generate the Manifest.mf file. You can modify the source code to make other fine adjustments.



Free free to contact me for .java file (Easier copy/paste :D)

Fix message manipulation

I have had a chance to work on some projects which involved minor modifications to fix messages on a repetative basis. Since this project was in C++ i wrote an initial header file to get the required result. later on out of curiosity i ported the same methods to java and bingo i have methods to work both in C++ and Java. The discussion which proceeds is related to methods and their usage and source code for the same.

Need:
The work involved parsing of FixSession.log files and making some generic changes to it (like replacing all client names and codes with cypher text or changing stock codes to dummp code and so on). This was done to take care of compliance and regulatory requirements. So, i worte two methods :
String getValueForTag(String fixMsg,String tag)
String replaceTagValue(String fixMsg, String tag, String replace)

Why all string manipulation? the reason was i was reading a file containing these messages and so string was the most logical choice.. only later did i realise that this is what makes the task easier when working with realtime systems. Time for some code now.... (pasting images as formatting gets lost on blogger mail me for full source code)

replaceTagValue[java]


replaceTagValue[C++]


getValueForTag[java]


getValueForTag [C++]


After you are done with FixMessage manipulation you need to recalculate the checksum value and the body length value before u send out this message on the network. (why is this done? well these checks are useful in determining if the message recieved is complete in all respects and if not a resend can be requested. This is vital in case of applications working in unreliable network connectivity) And to do this you can use the following methods:

For CheckSum Calculation C++ (Tag 10)



Java:


For BodyLength calculation (tag 9) C++:


Java:

Introduction to FIX messages

FIX (Financial Information exchange) is a protocol which allows user/traders/financial systems to automate the order management……. Enough !!! lets stop this technical blabbering and get something concrete and easy to digest on this plate. Its basically a set of rules just like formula one.

Is It a Data Representation Format?
Protocol is just a heavy name for rules, rules which we all love to hate which ironically make them indispensable. Before I start with any of this crap, let’s set up some sample data, using which we will try to understand why we need these rules in the first place. Consider an order with very small number of attributes. (The types which we love to create while creating sample orders)

Order Type: single order
Stock Code: 7200.T
Quantity: 10000
Price: 4.21

Whatever I have typed above can also be a way to represent an order, which can be easily interpreted by the human beings. But from an application perspective I can also represent the same data something like this:

As you would have guessed it by now this is an xml representation of the same data. The nice thing about xml is that its universally understood. So, in case you want to send out data from one application to another, one of the most popular ways of doing it is to send out an xml representation. But in spite of its many advantages, xml has some disadvantages. There is no standard on what tags can be used as anyone can use his/her own custom tags. Also xml data is difficult to parse when the hierarchy is very deep and adds additional overheads for applications which are meant to be hugely scalable.

Just for the sake of completeness I have showed below another form of data representation called JSON which is common in AJAX based web applications.

Bottom-line being that you can use any format for data representation, provided the “rules of the engagement are correctly specified” or in heavy words the “protocol is put in place”.

Another popular format is the tag/value representation, which is depicted something like this

10=A;11=7200.T;12=10000;13=4.21;

Where, 10 is Order Type
11 is stock code
12 is quantity
13 is price

This format has advantages, its crisp and the data is represented in just one line. There is no restriction on the number of attributes. And its simple “tag=value” separated by delimiters which you can select your own favourite. Personally I would prefer this nice isn’t it !!! But again there is no standard, no rules !!! I can define my own set of tags and make my own API.
So, all different organisations will have their own propriety tagging conventions (which is actually true in case of the client i work for) This is suitable for internal workings of one single organisation but obviously has loopholes when considering the scenarios when systems across organisations need to interact with one another.

Keeping all these considerations in mind people in the financial domain thought and agreed to another kind of data representation (which is basically the tag/value way of representation). This has its advantages. If you create an application which supports FIX protocol you can make your application “pluggable” to other third party systems/applications whether they are upstream or downstream.
In comes FIX to solve the problem. Let us have a look at the fix representation of the data we have been talking all this while.



If you look up the fix specification (http://fixprotocol.org/) following tags are of special importance and they form part of all the fix messages. They are:
8: This is the BeginString tag which identifies the beginning of new message and the protocol version. This is always the first tag in any fix message.
9: This is the BodyLength tag which contains the message length of the fix message. This is always the second tag of any fix message.
10: This is the CheckSum tag, which contains the basic checksum value of the entire message. This always forms the last tag of the FIX message and signifies the end of message.
The delimiter is the char 001 as per ASCII character set.

So, to conclude this section, we can safely say that FIX is just another format used to represent data, specific to financial sector.

*** We will be discussing these tags in more detail in later parts. Whenever in doubt which tag means what you can refer to the following online Fix dictionary: http://www.b2bits.com/fixopaedia/fixdic42/index.html

That’s enough for introduction !!!
So to conclude this section let me quote the official definition of FIX which says “The Financial Information Exchange protocol (FIX) is an open specification intended to streamline electronic communications in the financial securities industry. FIX supports multiple formats and types of communications between financial entities including email, texting, trade allocation, order submissions, order changes, execution reporting and advertisements.”

I hope the definition makes more sense now !!!

Autocomplete (YUI)

1. OVERVIEW

Web applications generally consist of forms, with textboxes, checkboxes, dropdowns etc to gather the user input. With textboxes the developer need to provide implementation for validations and with dropdowns he needs to provide the data from which the user can make a choice. But there is a small problem, the inconvenience caused to user in case the data is huge. For Example consider a dropdown with 10000 options.

Yahoo’s autocomplete provides an answer to that, by filtering the results based on the user input. The best part of the implementation lies in the ease of development using Ajax without worrying about the HttpServletRequest or the ActiveX object (for IE v4.0 and above) or the rendering of data in the jsp from the XML format it’s received in.

That brings with it huge advantages in terms of development turnaround time.

2. FUNCTIONALITY

Consider a scenario where the user needs to enter an item into a text box which could ambiguous or hard to remember and therefore has the potential to be mis-typed.
AutoComplete control fits into the requirements under the following set of requirements:
1. The suggestions can be pulled from a manageable set of data.
2. The input item can be entered in multiple ways.
3. The input item can be matched with a specific data item in the system.
4. Speed and accuracy of entry is an important goal.
5. The total number of items would be too large or inconvenient for displaying in a standard drop down box.

In case there is a situation where any of these conditions are present Autocomplete control is the way forward. With following advantages (User’s perspective)
1. Reducing the number of keystrokes allows for faster user input.
2. Showing additional formatting information in the suggested items list (like the email address in the example above) removes ambiguity.
3. Continual feedback helps the user narrow in on the correct choice.

Kindly refer to following section for more details:
http://developer.yahoo.com/ypatterns/pattern.php?pattern=autocomplete

3. BUSINESS CONTEXTS
In order to build a robust and efficient reusable component, it is essential to identify a variety of areas in which it can be applied. These could be
i) All struts based applications will work perfectly with this. For other kinds of web applications all you need to do is to configure your datasource to provide the data in XML format. It can be in any platform such as PHP scripts, Java applications, or web services. At its core a datasource is a URL which provides you with the required filtered data in the format of your choice.
iii) Any web based application with javascript as the scripting language can use this autocomplete functionality. Yes, it can work with .Net web applications as well.

4. DEPENDENCIES

To start using this control YUI.zip(Used v2.3.1) needs to be downloaded from the developer.yahoo.com site. The .js files on which there is dependency can be found in this folder (upon unzipping of course).
Additionally Struts related jar needs to be added (present in the lib folder)

5. IMPLEMENTATION PROCEDURE/CONFIGURATION

The implementation begins with creating a datasource to provide the data the user requires in XML format at runtime. This basically involves querying for the data from a web application over HTTP. Whatever be the implementation it all finally boils down to creation of a URL which accepts query in the following format:
http://url/?query=arg1&addnParams=args2
Once this datasource is in place, the struts action class (as per our implementation) can be used to get the data (from information systems). This data is then populated to an object of type intelliDataList, which in turn is set in the response object. This action is redirected to intelliList.jsp which writes the xml data after retrieving it from the intelliDataList object (this object has a function getXml() ). This forms the response text for the autocomplete widget, which gets parsed and is displayed.

Autocomplete widget is highly configurable. To control the look and feel the autocomplete.css needs to be modified suitably. Please refer to http://developer.yahoo.com/yui/articles/skinning/ section for a detailed discussion.
Similarly, the datasource configuration can be done very easily in the intellAutocomplete.js file. You can use three kinds of datasources (giving out plain text response, xml response orJSON), this POC uses the XML response.

**Note: Refer to Understanding Data Types and Schemas section at http://developer.yahoo.com/yui/autocomplete/#uri for more in depth information

For all those of you familier with Struts based web applications following is a brief outline of what needs to be done to get this thing off ground.

Directory Structure and other details
1. /struts-ajax : This is the root directory of the web application .All the jsp and Html files are stored here.
2. /struts-ajax/script : All the javascript files initializing the yahoo widgets or used for any other functionalities are kept here.
3. /struts-ajax/script/autocomplete : The javascript files given by yahoo which facilitates the autocomplete functionality.
4. /struts-ajax/css : All the style sheets are kept here
5. /struts-ajax/WEB-INF : This directory contains all resources related to the application that are not in the document root of the application.This is where your web application deployment descriptor is located.
6. /struts-ajax/WEB-INF/src/ajaxsample : This directory contains the java files gets all possible data entries for the a particular textbox.
7. /struts-ajax/WEB-INF/src/ajaxsample/controller : This directory contains the action class used to filter the entire list according to the text entered by the user.
8. /struts-ajax/WEB-INF/src/ajaxsample/form : This directory contains the Action forms corresponding to jsp.
9. /struts-ajax/WEB-INF/src/ajaxsample/model: This directory contains the two classes:
* IntelliData – A list containing objects of this class is expected from the user.
* IntelliDataList – A convenience bean that holds a list of IntelliData instances.
10. /struts-ajax/WEB-INF/classes : This directory is where class files of above mention java files are kept.
11. /struts-ajax/WEB-INF/lib : This directory contains Java Archive files that the web application depends upon. For example, this is where you would place a JAR file that contained all the Struts framework classes.

6. LIMITATIONS
One serious limitation arises when the data from the datasource contains some special characters which hinder the formation of corresponding XML. In that case the XML received by the widget is not well formed and the widget stops working.
This Reusable component has provided implementation for filter out such characters using the function getValidXML() which can be suitably overridden if necessary.
Also, when the amount of data being fetched is huge here can be a performance drop, even with the build in caching. To overcome this issue the size of the cache maintained by the widget can be increased, or caching service be implemented at the datasource level. This is especially true in case the data fetched is live data (not cacheable) or has to interact with

7. REFERENCE ARTIFACTS
Refer to the following online resource for further details, as this document draws heavily from the inputs provided therein.
http://developer.yahoo.com/yui/autocomplete

8. How does it all look:

Normal Text boxes:

Type search parameter to get result:

Type " " to get a complete listing:

The values in the Age column are constrained by the value selected in name: