THE AUSTRALIAN NATIONAL UNIVERSITY
FACULTY OF ENGINEERING AND INFORMATION TECHNOLOGY
DEPARTMENT OF COMPUTER SCIENCE
Development of Web Scope
(Using the Hibernate Package for
Meta-Data)
Le MA
(u4086599)
Semester 2, 2006
This is the final report for the COMP6703 eScience Project (Semester 2, 2006),
supervisors are Dr. Henry Gardner and Dr. Raju Karia.
TABLE OF CONTENTS
ACKNOWLEDGMENTS
ABSTRACT
1. INTRODUCTION................................................................................................. 4
1.1. OVERVIEW……………......................................................................................4
1.2. INTRODUCTION TO HIBERNATE..................................................................7
1.3. INTRODUCTION TO MDSPLUS........................................................................8
1.4. INTRODUCTION TO WEBSCOPE......................................................................10
1.5. INTRODUCTION TO Java Servlets and JSP......................................................11
1.6. INTRODUCTION TO Apache TomCat……......................................................12
1.7. INTRODUCTION TO Previous Project……......................................................12
2. REQUIREMENT ANALYSIS............................................................................15
2.1. SOFTWARE REQUIREMENTS...........................................................................15
2.2. OPERATING SYSTEM……….............................................................................15
2.3. LANGUAGES……………...................................................................................15
2.4. CLIENT REQUIREMENTS.................................................................................16
3. SCHEDULING……………………………….....................................................17
3.1. PLANNED TIME TABLE ………………………………...........................................17
3.1. Actual TIME TABLE ………………………………................................................18
4. MODELING………..............................................................................................20
4.1. DATA BASE DESIGN …….……………………………............................................20
4.2 Key Methods and relevant Algorithms …...…….............................................22
4.3 State machine for the “metadata” classes.………….……………………................23
5. IMPLEMENTATION...........................................................................................26
5.1 Writing prototypes of each parts of the system based on the requirements
………….……………………................26
5.2 Core techniques …….……………………………...................................................26
6. CONCLUSION, DISSCUSSION AND FUTURE WORK…….......................29
6.1 Design Discussion….……………………………....................................................29
6.2 Key API commands.…………………………….....................................................30
6.3 Architectural Consideration …………………......................................................31
6.4 Testing ………………..……………………….....................................................32
7. CONCLUSION AND FUTURE WOR ............................................................35
7.1. CONCLUSTION.............................................................................................35
7.2. FUTURE WORK ............................................................................................35
8. REFERENCE .......................................................................................................35
8.1. BOOKS................................................................................................................35
8.2. Internet References ........................................................................................... 36
APPENDIX A GANTT CHART...............................................................................37
APPENDIX B USER GUIDE……………................................................................38
2
Acknowledgments
During the whole process of my project – Development of Web Scope, my
supervisors Dr. Henry Gardner and Dr. Ruju Karia have given me endless help from
collecting system requirements, making efficient schedule; and later during the
implementation stage, my supervisors gave me the most important technical support
and advice. I have received unselfish help from the former project student Ajith
Mannanakunnel Jose, he came at his spare time and gave me fully introduction of the
former project system, and quite useful technical help.
Also, I have to thanks Hugh Fisher who installed the computer which I am working
on and the required software for the project.
Abstract
This report describes the project “Development of Web Scope” of eScience
department of the Australian National University. This project is an extension to the
work done by the former eScience students Shi Hu in semester1 2005 and Ajith
Mannanakunnel Jose in semester2 2005.
The main functional goal of the whole project is retrieving the nuclear fusion datasets
from the MDSPlus based server and contribute metadata which contains the
information of the relevant data retrieved from the sever dynamically back through a
web browser. The highlight feature of this project is to use Java Servlets, JSP and Java
Hibernate package within the TomCat to generate, save and retrieve metadata
dynamically.
3
1 Introduction
1.1 Overview
1.1.1 Background
The general idea of doing this whole project is to set up a web based system which
can allow user around the globe to retrieve the data from the MDSPlus database
through the internet and contribute some metadata based on the retrieved datasets
back to the sever side database in order to share the knowledge.
The previous project which had been done by students Shi Hu and Ajith
Mannanakunnel Jose had already developed the Web Scope which can connect to the
MDSPlus server and retrieve datasets from the MDSPlus sever, also cache the
retrieved datasets at the client side by using the Java object relational mapping
solution “Hibernate”.
At present the researchers cannot contribute to the datasets which they have their own
opinions or some kinds of suggestions, they can not send metadata back to the server
database. Furthermore, different researchers may require creating different forms of
metadata based on the original datasets.
The project is an attempt to resolve the existing issues with dynamically contributing
the metadata back to the MDSPlus server. The highlight of this project is the use of
Java object relational mapping solution “Hibernate” together with JAVA Servlets and
JSP to solve the dynamically operating metadata issues with current data retrieval
system.
1.1.2 Purpose
1.1.2.1 User Side
The purpose of doing this project is to enable users to generate, save and retrieve
metadata dynamically by using the Web Scope application.
1.1.2.2 Technical Aspect
The purpose of doing this project is to enable the attempt of writing, compiling and
loading Java class dynamically, and implement Java’s dynamic ability in reality.
1.1.2.3 Reasons
The reason for adding these functions to the existing project is :
• To allow client to create the format of the metadata which they are going to
contribute to the database by themselves, it provides wide freedom to the
client.
4
• To change the traditional way of writing and compiling Java classes.
• To attempt to load Java classes at runtime by calling the Java class loader.
1.1.3 Target results and project clients
1.1.3.1 Target results
The “Development of Web Scope” project has three target results besides several
benefits that may flow through. They are:
• Allowed client to create metadata table dynamically
• Allowed client to save metadata dynamically
• Allowed client to retrieve the saved metadata dynamically
The above mentioned target outcomes can help the project clients in the following
way.
Clients can create metadata which are based on the data retrieved from the MDSPlus
dynamically by using the dynamic metadata table, the metadata can be shared by the
others who review the same node in the MDSPlus, the clients can share useful
knowledge from each other by creating or reviewing metadata based on the data from
MDSPlus. All clients can contribute metadata as they want and do not need to follow
strict format. This would improve the quality of the metadata which was saved to the
sever database.
1.1.3.2 Deliverables
The project deliverables are mentioned in the table below:
Deliverable Due Date
Initial Presentation 5th August 2006
Software Demonstration November 2006
Final Presentation November 2006
Source Code Submission November 2006
5
1.1.3.3 Customers
The project customers who are ensuring the success of achieving target outcomes are
nuclear fusion researchers or scientists from different parts of the globe.
1.1.4 Stakeholders
The stakeholders of this project are described below.
Stakeholder Name Contact Details
Supervisor Mr. Raju Karia rjkaria@smartchat.net.au
Supervisor Dr. Henry Gardner Henry.Gardner@anu.edu.au
Developer Mr. Le Ma xiaomagetony@yahoo.com.au
6
1.2 Introduction to Hibernate
Hibernate is a powerful, high performance object/relational persistence and query
service.
1.2.1 Dual – Layer Cache Architecture
Hibernate applies Dual-layer Cache Architecture, which means there are two level of
cache in Hibernate, the JVM cache and the Session cache. Through these two levels
of cache the data caching process become more efficient and faster. This feature was
implemented completely by the former student – Ajith.
1.2.2 Transparent Persistence
Hibernate can create tables dynamically based on the XML mapping file which
contains the information of the class relative to the table and the class file which sets
up the content of the table. Also, it allows saving data to the database and retrieves
data from the databse.
1.2.3 Query options
Hibernate offers plenty of query options, developers can write plain SQL, object-
oriented HQL(Hibernate Query Language), programmatic Criteria and example
queries. It is quite flexible for developers, and powerful for querying.
1.2.4 Cooperate with Java Servlets
Hibernate cooperates with Java Servlets within the Apache TomCat to set up the
communication between the client and web server. There would be many Hibernate
sessions during one servlets session, the Hibernate session can fulfill user’s one or
more action during the specific user service. For example, when user required to set
up one table in order to save data into database, the servlets session will collect the
user’s information and the relevant web information, and the Hibernate session will
set up the table based on the XML mapping file and the class file by being called
within the servlet session.
7
1.3 Introduction to MDSPlus
1.3.1 A set of tools
MDSPlus is a set of tools for data acquisition and storage. A typical data acquisition
cycle might involve:
• Initialize
o copy model file to shot file
o find all the action descriptions and order them into a dispatch table
o dispatch all initialization actions
• Pulse
o fire high-speed timing system which runs experiment and data
acquisition hardware
• Store
o dispatch all store actions
o dispatch all analysis actions
1.3.2 A methodology
MDSPlus is a methodology for management of complex scientific data.
1.3.3 Data type
MDSPlus supports a rich set of primitive and composite data type, The simple data
types include signed and unsigned integers (1, 2, 4, 8, and 16 Bytes), single and
double precision real numbers, single and double precision complex numbers, and
character data. Arrays of these data types with up to 7 dimensions can be represented.
The most important composite types include: SIGNALS, which contain data plus
associated independent axes (eg. temperature vs space and time); and DEVICES,
which are used to associate setup parameters, actions (task descriptions), and data for
data acquisition or automated analysis. MDSplus devices provide a well structured
mechanism for implementing data driven applications.
1.3.4 Client/Server model
A user with appropriate permissions, can access data from an MDSplus server
anywhere on the internet. Access is "service" rather than file oriented. Data on remote
servers is read directly from within applications, no file transfers are used.
1.3.5 Development
The MDSPlus is developed jointly by the Massachusetts Institute of Technology, the
Center for Nuclear Resarch (Padua, Italy), and the Los Alamos National Lab,
MDSplus is the most widely used system for data management in the magnetic fusion
energy program. It is currently installed at over 30 sites spread over 4 continents.
8
1.4 Introduction to Web Scope
The “Web Scope” is a new system proposed by my client Raju Karia and supervisor
Dr.Henry Gardner. It was first used by the former project student Ajith. The following
diagram describes the “Web Scope”.
As described in the figure above, the Web Scope is running on a web server – the
TomCat, and it maintains the HSQLDB, the Hibernate database.
1.5 Introduction to Java Servlets and JSP
Servlets are Java classes run by a Web server which has an interpreter that supports
the Java Servlet specification. Java Servlets can used for these purpose:
• Collecting user input from a form on a Web page
• Receiving information from arguments specified as part of a URL
• Running programs on the computer that runs the Web server
• Storing and retrieving cookies, files that store a user’s preferences and other
information on his computer
• Sending data back to a user in the form of an HTML document, GIF graphic,
or another format
Java Servlets have several advantages in the following aspects:
9
• Proving a standard way to extend web servers to support dynamic content
generation
• Offering full support for sessions, a way to keep track of a particular user over
time as a Web site’s pages are being viewed
• Having the same portability as the Java language
• Running efficiently in memory
Java Servlets work in the following way:
When a client makes a request involving a servlet, the server loads and executes the
appropriate Java classes. Those classes generate content, and the server sends the
content back to the client. The most specific difference to others such as CGI script, is
– persistence. Instead of shutting down at the end of each request, the servlet can
remain loaded, ready to handle subsequent requests. The lifecycle is :
All of these approaches which come from the Java Servlets can be enhanced by
JavaServer Pages (JSP). JSP is a way to create Web pages that mix static HTML with
the output of servlets and Java expressions. Using JSP, Java code can be placed on
Web pages among HTML markup and edited like any other page.
1.6 Introduction to Apache TomCat
Apache TomCat is a web container developed at the Apache Software Foundation
(ASF). Tomcat implements the servlet and the JavaServer Pages (JSP) specifications
from Sun Microsystems, providing an environment for Java code to run in
cooperation with a web server.
10
In this project, Apache Tomcat is used as the web container.
1.7 Previous Project
1.7.1 Current Way of Retrieving Data from MDSPlus
The researchers use software like “EScope” or “JScope” to connect to the MDSPlus
server and retrieve the required data. The major steps involved in this data retrieval
process are as follows:
• Researchers connect to the MDSPlus server, and requests data sets using client
side application called “EScope”.
• Based on the “EScope” client’s request the MDSPlus retrieves datasets from
its own data storage mechanism.
• MDSPlus server sends the retrieved datasets to the “EScope” client.
• “EScope” Client displays the datasets as graphs.
1.7.2 Shi Hu’s Project
The first attempt of the project was done by student Shi Hu in semester 1, 2005. The
whole structure is as below:
11
This system works in the following way:
• Data from the MDSPlus server is initially downloaded from the MDSPlus
server using an application called “Importer” and stored in the local HSQL
database.
• An application called “DBserver” is used to serve the requests from the
clients (EScope). Based on the client requests the datasets from the HSQL
database is returned to the clients.
• Using client application called “EScope viewer” a connection is established
with the “DB Server“, and retrieves the required dataset from the local
database (HSQLDB) associated with it. EScope client plots graphs based on
the retrieved datasets.
1.7.3 Ajith’s Project
The second attempt of the project was done by previous student Ajith Mannanakunnel
Jose, the whole structure is as below:
12
This system works in the following way:
• The original data is stored in MDSPlus Server and the retrieved data is stored
in HSQLDB for later requirement.
• Web Scope is used to serve the requests from the clients. Based on the client
requests, it will search from the local database – HSQLDB; if the required data
does not exist in the HSQLDB, it will send the requests to the MDSPlus
Server and retrieve data back to HSQLDB.
• Web Browser is used to show the applications of the system and allow client
to input requests and review datasets also with the applets.
13
2 Requirement Analyses
2.1 Software Requirements
The main software and other tools used for the development of “Web Scope” are
given below.
• Jakarta Tom Cat 5.05
• Hibernate3.1
• JDK1.5
• HSQLDB
• FireFox (or any Java Applet enabled Web Browser)
• Emacs Editor
2.2 Operating System
Os Name Version
Linux Debian
2.3 Languages
14
2.4 Client Requirements
The following table shows the major requirements specified by the client.
No. Name Description Priority
R1 Using Hibernate
The “Web Scope” application should use the
benefits of “Hibernate” to efficiently cache the
datasets retrieved from MDSPlus.
HIGH
R2 Using Web Browser
Allow users to connect to MDSPlus server
and retrieve the required datasets using a
web browser.
HIGH
R3
Dynamically
creating
metadata table
Allow users to create metadata table
dynamically based on the specific
requirement.
HIGH
R4 Saving metadata Allow users to save metadata into the HSQLDB. HIGH
R5 Viewing metadata
Allow users to view all the user contributed
metadata. HIGH
R6 Platform independence
The “Web Scope” application should work
from all the major platform. HIGH
15
3 Scheduling
3.1 Planned Time Table
The whole project was divided to five different phases and specific time slots were
allotted for each of the phases. The following table shows the time table scheduled
initially for the project.
Phase Date Expected Duration(days) Tasks Notes
1 17
th Jul-
6th Aug 21
9 Understanding
requirements
9 Prepare for first
presentation
z Choose and meet
supervisor
z Understand
requirements
z Decide tools and
install software
on the computer
z Study old reports
z Make slides for
first presentation
2
7th Aug
– 27th
Aug
21 Modeling
z Analyze existing
project done by
Ajith
z Modeling and
design
architecture for
the new project
z Learn various
software
3 28
th Aug
– 8th Oct 48 Implementation
z Do coding,
Testing and
Debugging
4 9
th Oct –
22nd Oct 14
Documentation and final
report creation
z Complete all
documents
z Write the final
report
5
23rd Oct
– 29th
Oct
7 Preparation for final presentation
z Edit slides for
final persentation
16
3.2 Actual Time Table
The following table shows the actual time table for the project.
Phase
Date Expected Duration(days) Tasks Notes
1 17
th Jul-
6th Aug 21
9 Understanding
requirements
9 Prepare for first
presentation
z Choose and meet
supervisor
z Understand
requirements
z Decide tools and
install software
on the computer
z Study old reports
z Make slides for
first presentation
2
7th Aug
– 2nd
Aug
28 Modeling
z Analyze existing
project done by
Ajith
z Modeling and
design
architecture for
the new project
z Learn various
software
3
3rd Aug
– 15th
Oct
48 Implementation
z Do coding,
Testing and
Debugging
4
16th Oct
– 29th
Oct
14 Documentation and final report creation
z Complete all
documents
z Write the final
report
5 30
th Oct
– 5th Oct 7
Preparation for final
presentation
z Edit slides for
final presentation
The major issues faced during the project are described below:
• The whole project was divided into two major parts – echo2 and dynamic
metadata, and my task is changed from the whole project to the second part –
dynamic metadata.
17
• It took one week to set up the project machine (the computer with Linux) and
two weeks to get the key of the student project room. So, one week was added
to the modeling part, due to the function check and understand.
• The project is an extension of previous students’ project, so, it took not much
time on the modeling part, but it took long time to understand the function of
each part of the system.
• Hibernate, Java Servlets and JSP are quite new to me, so it also took time to
learn those new techniques during the whole project.
• The major issue during the implementation part is that how to find out and use
these methods or classes of Hibernate and Java. For example, the Java
compiler at runtime, the “Javac.compile()” which is included in the “Method”
class of Java package. Also, the usage of the method.
18
4 Modeling
4.1 Database Design
The main database structure is inherited from the former project (please refer to
appendix A for more information of the database), and the new table which was added
to the database are shown below:
Table Name Explanation
DYNAMICTABLE
Store all detail of the table which user
dynamically created base on the retrieved
datasets.
METADATA Store all metadata information contributed by the user based on the retrieved datasets
“Hibernate” is used to handle all interactions with the database. The process is as
below:
1. create Java persistent classes represent the database tables.
2. map Java classes into the Hibernate mapping file (please refer to the
appendix B form more information).
3. Hibernate dynamically creates the tables based on the mapping file, and
performs relevant actions, such as save data into database or retrieve data
from database.
These two new tables are different from the originals because both of them are created
dynamically when the application is running. The whole process of creating these
tables is as below:
1. user inputs relevant information to the application.
2. Java persistent classes are created and compiled dynamically after the
input action.
3. Java persistent classes information is added to the Hibernate mapping
file dynamically.
4. Java persistent classes are loaded by the JVM dynamically
19
5. Hibernate dynamically creates the tables.
All the tables in the database have an id field as primary key and the values are
dynamically generated by the Hibernate package.
Because the “DYNAMICTABLE” and “METADATA” table are generated
dynamically during the runtime of the application, so they are not exist until user acts
the corresponded operations. But once they are created, there are several reference
variables are set up into the table, the relationship among these tables are as below:
Has a
1
NODE
Id: long
Experiment: String
Shot:int
Sever: String
Port: int
nodePath: String
havaGraph: Boolean
METADATA
Id : long
Node : long
User : long
Dynamic field
Due to the “METADATA” table is created dynamically, and the variables which will
be involved into the table are unknown, so we just put the “Dynamic field” into the
table to refer to these variables. Also, this is the exact way we try to satisfy every user,
and to let users to contribute the metadata in the way they want.
0 .. *
Linked to
0 .. *
Linked to
Has a
1
USERS
Id: long
Institution:String
Address: String
Email: String
Password: String
Education: String
20
4.2 Key Methods and relevant Algorithms
Java ClassLoader
Class loaders are a powerful mechanism for dynamically loading software
components on the Java platform. Class loaders existed in the first version of the Java
Development Kit (JDK 1.0). The original purpose was to enable applet class loading
in the HotJava browser. Since that time, the use of class loaders has been extended to
handle wider range of software components.
The purpose of using class loaders is to support dynamically loading of software
components on the Java platform. The unit of software distribution is a class. Classes
are distributed using a machine-independent, standard, binary representation known as
the class file format (such as metadata.class of this project). Class files are produced
by Java compilers, and can be loaded into any Java virtual machine. The Java virtual
machine executes the byte code stored in class file.
In this project, most of class files were produced by the Java compilers before the web
scope was running. It means that Java files were created by programming code, and
then were compiled manually by using the command line (such as Javac ..). After that,
the class files were produced and then were put into the location of Aapach TomCat.
When the web scope application is running, the class files can be loaded automatically.
However, those class files which were produced dynamically during the application
can not be loaded by the Java virtual machine automatically. So, we need call the
class loader to load those classes to be used by the application.
The method of loading classes is ClassLoader.loadClass(), it is the entry point to the
ClassLoader. Its signature is as follows:
Class loadClass(String name, Boolean resolve);
The “name” parameter specifies the name of the class that the Java virtual machine
needs.
21
The “resolve” parameter tells the method whether or not the class needs to be resolved.
It is a flag to tell the class loader that classes referenced by this class name should be
resolved (that is, any referenced classes should be loaded as well)or not.
4.3 State machine for the “metadata” classes
The “metadata” classes were produced dynamically during the web scope running
time, they were created by users, and they would contain the information of the
metadata which users want to contribute to the database based on the current datasets
they were retrieving. The state machine of these classes are as follow:
Note: because the class name is defined by different users, so we can not give the
exact name of the class, we use “newMetaData” to refer to the class.
22
1. Creating Java file
fielName= = Tablename;
loop
for (i=0; i"+makeJavaScript(colNumber)+""+title+"