Know more and connect with me on [Linkedin Profile].

Tuesday, December 26, 2006

Last year I restarted my dream again, Ubuntu!

I really fond of open source software, there are several reason, stating it again is a repetition, so I will just put some links:

Several years ago, I have tried Sun Solaris, Free BSD, Redhat 7.0 and Mandrake, they all was nice, but I did not continue using it, I already believe in the concept of open source but the products was not suitable to me as I am using a lot of third party Windows Desktop software that relates to my work and I haven't alternatives.

Although this, I enjoyed learning java and using it everywhere where possible, from web sites to desktop applications, even in
inhouse tools to manage issues on some servers.

Last year I restarted my dream again to use Linux instead of Windows, I have tried Centos, fedora core 5 and finally fall in love with Ubuntu, I am really a fan of Ubuntu, but after sometime the same issue of third party Windows software is taking me from Ubuntu, a really bad situation, I have tried Wine emulation, it is good, but not good enough, I hope I can contribute to the project to improve it but I am so busy.

I found a very good option that seems to work with me perfectly. It is Cygwin! I have to admit that, in recent years it was clear to me that the command line tools and shell programming is extremely powerful, in many cases much more than the GUI tools. So the idea is to use Cygwin command line tools and shell scripts whenever it makes sense. Some tasks are suitable for command line like using subversion in large projects and and automating my local files update to backup server through rsync.

So at home and work I am using Ubuntu as much as I can, when I have to use Windows I use open source software and the great cygwin linux layer as much as I can.

- Cygwin
- Firefox
- Thunderbird to check email and RSS feeds.
- Subversion, it is programmers version control system.
- RapidSVN, a client tool to subversion.
- Programmers notepad, simple and fast editor
- Eclipse, Java and C++ IDE.
- Netbeans, java develoment IDE.
- 7Zip, a great tool that open even rar file.
- Active Perl.
- DIA diagramming tool.
- Open Office.
- PSI, jabber based chat client.
- WinMerge, to show file directories differences and make merge.
- StarDict , an open source dictionary

If you want any of these tools, just google using thier name, and Google will present it as the first hit, thanks to Google.

If you want my support to start moving to open source, just give me the honor to help you, anyway I a going to post some tips that helps me in the open source world.

Sunday, December 24, 2006

The Simplest Logging Class in C++

I want to create the simplest ever logging class in C++, so I created a single Log.h file and created a sample application that uses it.
This was very useful in the small application I am working on.

It compile with gcc and it works as expected.

known problems:
the Log.write() function may make problems in multithreaded applications, in such case you must put a critical section around it.

You are completely free to use the code as you wish. No limits.

//logging header file
#include <stdio.h>
#include "log.h"


int main()
//initialize file logger

LOG_MSG("Application initialization");

printf("This is the application\n");

LOG_MSG("Application end");


Look at the Log.h file


#ifndef __LOG_H
#define __LOG_H
//It is bad to include files within other include files, but this is an exception to simplify the code
#include <stdio.h>

//must be put in global scope and used once in all files
#define LOG_DECLARE Log __log;
//used in any file need to access the log files, except the file that has LOG_DECLARE statment
#define LOG_USE(); extern Log __log;
//init log files, called once in program initialization
#define LOG_INIT(fileName) do{if(fileName!= NULL)__log.init(fileName); else printf("error init log file %s", fileName);}while(0);

//here are the different levels of logging
#define LOG_VERBOSE(msg) __log.write("verbose", msg, __FILE__, __LINE__); //detaild info
#define LOG_MSG(msg) __log.write("msg", msg, __FILE__, __LINE__); //brief info
#define LOG_WARN(msg) __log.write("warn", msg, __FILE__, __LINE__); //warning
#define LOG_ERR(msg) __log.write("error", msg, __FILE__, __LINE__); //error
#define LOG_FATAL(msg) __log.write("fatal", msg, __FILE__, __LINE__); //fatal error

class Log
FILE *fp;
bool logOk;


{fp = NULL;}
~Log() //the destructor closes the file

void init(char *pfileName)
{ if(pfileName != NULL) {fp = fopen(pfileName, "a+"); if(fp != NULL) fseek(fp, 0, SEEK_END);} }

void close()
{if(fp != NULL) fclose(fp); fp = NULL;}

//FIXME: A critical section is required to protect the file writing function in multithreading programms
void write(char *pType, char *pMsg, char *pFileName, int lineNo)
{ if(fp != NULL) fprintf(fp, "%s\t%s\t%s\t%s\t%d\t%s\n", pType, __DATE__, __TIME__, pFileName, lineNo, pMsg); }


Saturday, December 23, 2006

Web Load/Stress/Stability and Performance Test Tools

Microsoft's Web Application Stress Tool provides an easy way to simulate large numbers of users against your Web application. This tool makes it possible to make intelligent decisions about hardware and software load incurred by your application and how much traffic a given machine or group of machines can handle. In this article Rick shows how the tool works and how to properly interpret the performance data it generates.

Look at this tutorial:

In my experience this tool has many bugs and wasted a lot of our time, try it but if you have problems consider researching other tools, here is a long list of available tools.

Already, there is a list of web stress tools at
The URL has useful tools that I copy/paste it here, I just afraid the URL may disappear!

Load and Performance Test Tools

* The Grinder - A Java load-testing framework freely available under a BSD-style open-source license. Orchestrate activities of a test script in many processes across many machines, using a graphical console application. Test scripts make use of client code embodied in Java plug-ins. Most users do not write plug-ins themselves, instead using one of the supplied plug-ins. Comes with a mature plug-in for testing HTTP services, as well as a tool which allows HTTP scripts to be automatically recorded.

* Proxy Sniffer - Web load and stress testing tool from from Ingenieurbüro David Fischer GmbH Capabilities include: HTTP/S Web Session Recorder that can be used with any web browser; recordings can then be used to automatically create optimized Java-based load test programs; automatic protection from "false positive" results by examining actual web page content; detailed Error Analysis using saved error snapshots; real-time statistics.

* Funkload - Web load testing, stress testing, and functional testing tool written in Python and distributed as free software under the GNU GPL. Emulates a web browser (single-threaded) using webunit; https support; produces detailed reports in ReST, HTML, or PDF.

* Avalanche - Load-testing appliance from Spirent Communications, designed to stress-test security, network, and Web application infrastructures by generating large quantities of user and network traffic. Simulates as many as two million concurrently-connected users with unique IP addresses, emulates multiple Web browsers, supports Web Services testing Supports HTTP 1.0/1.1, SSL, FTP, RTSP/ RTP, MS Win Media, SMTP, POP3, DNS, Telnet, and Video on Demand over Multicast protocols.

* Loadea - Stress testing tool runs on WinXP; free evaluation version for two virtual users. Capture module provides a development environment, utilizes C# scripting and XML based data. Control module defines, schedules, and deploys tests, defines number of virtual users, etc. Analysis module analyzes results and provides reporting capabilities.

* LoadManager - Load, Stress, Stability and Performance testing tool from Alvicom. Runs on all platforms supported by Eclipse and Java such as Linux, Windows, HP Unix, and others.

* QEngine Web Performance Testing - Automated testing tool from AdventNet for performance testing (load and stress testing) of web applications and web services. For Linux and Windows.

* TestLOAD - An automated load testing solution for IBM iSeries from Original Software Group Ltd. Rather than placing artificial load on the network, it runs natively on the server, simulating actual system performance, monitoring and capturing batch activity, server jobs and green-screen activity. For web and other applications.

* NeoLoad - Load testing tool for web applications from Neotys with clear and intuitive graphical interface, no scripting/fast learning curve, clear and comprehensive reports and test results. Can design complex scenarios to handle real world applications. Features include data replacement, data extraction, SOAP support, system monitoring (Windows, Linux, IIS, Apache, WebLogic, Websphere...), SSL recording, PDF/HTML/Word reporting, IP spoofing, and more. Multi-platform: Windows, Linux, Solaris.

* webStress - Load and stress testing service from MoniForce BV. Includes recommendations on how to fix performance-related problems.

* Test Complete Enterprise - Automated test tool from AutomatedQA Corp. includes web load testing capabilities.

* WebPartner Test and Performance Center - Test tool from WebPartner for stress tests, load performance testing, transaction diagnostics and website monitoring of HTTP/HTTPS web transactions and XML/SOAP/WSDL web services.

* QTest - Web load testing tool from Quotium Technologies SA. Capabilities include: cookies managed natively, making the script modelling phase shorter; HTML and XML parser, allowing display and retrieval of any element from a HTML page or an XML flux in test scripts; option of developing custom monitors using supplied APIs; more.

* LoadDriver - Load test tool from Inforsolutions emphasizes ease of use; directly drives multiple instances of MSIE, rather than simulating browsers. Supports browser-side scripts/objects, HTTP 1.0/1.1, HTTPS, cookies, cache, Windows authentication. Tests can be scriptlessly parameterized with data from text files or custom ODBC data source, for individual userID, password, page to start, think times, data to enter, links to click, cache, initial cache state, etc.

* Test Perspective Load Test - Do-it-yourself load testing service from Keynote Systems for Web applications. Utilizes Keynote's load-generating infrastructure on the Internet; conduct realistic outside-the-firewall load and stress tests to validate performance of entire Web application infrastructure.

* SiteTester1 - Load test tool from Pilot Software Ltd. Allows definition of requests, jobs, procedures and tests, HTTP1.0/1.1 compatible requests, POST/GET methods, cookies, running in multi-threaded or single-threaded mode, generates various reports in HTML format, keeps and reads XML formatted files for test definitions and test logs. Requires JDK1.2 or higher.

* httperf - Web server performance/benchmarking tool from HP Research Labs. Provides a flexible facility for generating various HTTP workloads and measuring server performance. Focus is not on implementing one particular benchmark but on providing a robust, high-performance, extensible tool. Available free as source code.

* NetworkTester - Tool (formerly called 'NetPressure') from Agilent Technologies uses real user traffic, including DNS, HTTP, FTP, NNTP, streaming media, POP3, SMTP, NFS, CIFS, IM, etc. - through access authentication systems such as PPPOE, DHCP, 802.1X, IPsec, as necessary. Unlimited scalability; GUI-driven management station; no scripting; open API. Errors isolated and identified in real-time; traffic monitored at every step in a protocol exchange (such as time of DNS lookup, time to logon to server, etc.). All transactions logged, and detailed reporting available.

* WAPT - Web load and stress testing tool from SoftLogica LLC. Handles dynamic content and HTTPS/SSL; easy to use; support for redirects and all types of proxies; clear reports and graphs.

* Microsoft Application Center Test - Tool for stressing Web servers and analyzing performance and scalability problems with Web applications, including ASP, and the components they use. Supports several authentication schemes and SSL protocol for use in testing personalized and secure sites. The programmable dynamic tests can also be used for functional testing. Visual Studio .NET Edition.

* OpenLoad - Affordable and completely web-based load testing tool from OpenDemand; knowledge of scripting languages not required - web-based recorder can capture and translate any user action from any website or web application. Generate up to 1000 simultaneous users with minimum hardware.

* ANTS - Advanced .NET Testing System from Red Gate Software. A load and stress testing tool focused on .NET web applications, including XML Web Services. ANTS generates multiple concurrent users via recordable Visual Basic .NET scripts and records the user experiences, at the same time performance counter information from Windows system is integrated into the results.

* Apache JMeter - Java desktop application from the Apache Software Foundation designed to load test functional behavior and measure performance. Originally designed for testing Web Applications but has since expanded to other test functions; may be used to test performance both on static and dynamic resources (files, Servlets, Perl scripts, Java Objects, Data Bases and Queries, FTP Servers and more). Can be used to simulate a heavy load on a server, network or object to test its strength or to analyze overall performance under different load types; can make a graphical analysis of performance or test server/script/object behavior under heavy concurrent load.

* TestMaker - Free open source utility maintained by and Frank Cohen, for performance, scalability, and functional testing of Web application. A framework and utility to build and run intelligent test agents that implement user behaviors and drive the system as users would. Features an XML-based scripting language and library of test objects to create test agents. Includes capability to check and monitor email systems using SMTP, POP3, IMAP protocols. Java-based tool - runs on any platform.

* SiteStress - Remote, consultative load testing service by Webmetrics. Simulates end-user activity against designated websites for performance and infrastructure reliability testing. Can generate an infinitely scalable user load from GlobalWatch Network, and provide performance reporting, analysis, and optimization recommendations.

* e-Load - Web load test tool from Empirix can simulate hundreds or thousands of concurrent users; accessible via a Web browser interface.

* Siege - Open source stress/regression test and benchmark utility; supports basic authentication, cookies, HTTP and HTTPS protocols. Enables testing a web server with a configurable number of concurrent simulated users. Stress a single URL with a specified number of simulated users or stress multiple URL's simultaneously. Reports total number of transactions, elapsed time, bytes transferred, response time, transaction rate, concurrency, and server response. Developed by Jeffrey Fulmer, modeled in part after Lincoln Stein's, but allows stressing many URLs simultaneously. Distributed under terms of the GPL; written in C; for UNIX and related platforms.

* JBlitz - Load, performance and functional test tool from Clan Productions. Runs multiple concurrent virtual simulate heavy load. Validates each response using plain text or regular expression searches, or by calling out to your own custom code. Full Java API. For testing and 'bullet-proofing' server side software - ASPs, JSPs, servlets, EJBs, Perl / PHP / C / C++ / CGI scripts etc.

* WebServer Stress Tool - Web stress test tool from Paessler AG handles proxies, passwords, user agents, cookies, AAL.

* Web Polygraph - Freely available benchmarking tool for caching proxies, origin server accelerators, L4/7 switches, and other Web intermediaries. Other features: for high-performance HTTP clients and servers, realistic traffic generation and content simulation, ready-to-use standard workloads, powerful domain-specific configuration language, and portable open-source implementation. C++ source available; binaries avail for Windows.

* OpenSTA - 'Open System Testing Architecture' is a free, open source web load/stress testing application, licensed under the Gnu GPL. Utilizes a distributed software architecture based on CORBA. OpenSTA binaries available for Windows.

* PureLoad - Java-based multi-platform performance testing and analysis tool from Minq Software. Includes 'Comparer' and 'Recorder' capabilities, dynamic input data, scenario editor/debugger, load generation for single or distributed sources.

* ApacheBench - Perl API for Apache benchmarking and regression testing. Intended as foundation for a complete benchmarking and regression testing suite for transaction-based mod_perl sites. For stress-testing server while verifying correct HTTP responses. Based on the Apache 1.3.12 ab code. Available via CPAN as .tar.gz file.

* Torture - Bare-bones Perl script by Lincoln Stein for testing web server speed and responsiveness and test stability and reliability of a particular Web server. Can send large amounts of random data to a server to measure speed and response time of servers, CGI scripts, etc.

* WebSpray - Low-cost load testing tool from CAI Networks; includes link testing capabilities; can simulate up to 1,000 clients from a single IP address; also supports multiple IP addresses with or without aliases. For Windows.

* eValid - Web test tool from Software Research, Inc that uses a 'Test Enabled Web Browser' test engine that provides browser based 100% client side quality checking, dynamic testing, content validation, page performance tuning, and webserver loading and capacity analysis.

* WebPerformance Load Tester - Load test tool emphasizing ease-of-use, from WebPerformance Inc. Supports all browsers and web servers; records and allows viewing of exact bytes flowing between browser and server; no scripting required. Modem simulation allows each virtual user to be bandwidth limited. Can automatically handle variations in session-specific items such as cookies, usernames, passwords, IP addresses, and any other parameter to simulate multiple virtual users. For Windows, Linux, Solaris, most UNIX variants.

* WebSuite - A collection of load testing, capture/playback, and related tools from Technovations for performance testing of web sites. Modules include WebCorder, Load Director, Report Generator, Batch, Manager, and others. WebSizr load testing tool supports authentication, SSL, cookies, redirects. Recorded scripts can be modified manually. For Windows.

* FORECAST - Load testing tool from Facilita Software for web, client-server, network, and database systems. Capabilities include proprietary, Java, or C++ scripting; windows browser or network recording/playback. Network traces can also be taken from over 15 third party tracing tools. Virtual user data can be parameterized. Works with a wide variety of platforms.

* e-Load - Load test tool from Empirix Software; for use in conjunction with test scripts from their e-Tester functional test tool. Allows on-the-fly changes and has real-time reporting capabilities. Includes script editor with advanced debugging and maintenance capabilities. Works with a wide variety of platforms.

* http-Load - Free load test application from ACME Labs to generate web server loads, from ACME Software. Handles HTTP and HTTPS; for Unix.

* QALoad - Compuware's QALoad for load/stress testing of web, database, and char-based systems. Integration with other Compuware tools provides an in-depth view by monitoring its operating system, database and network components, as well as the application itself. Works with a variety of databases, middleware, ERP.

* Microsoft WCAT load test tool - Web load test tool from Microsoft for load testing of MS IIS servers; other MS stress tools also listed.

* Portent Web Load test tool -'s low-priced web load testing tool. Has minimal hardware requirements. Page validation via matching string in page. Written in Java; multi-platform.

* SilkPerformer - Enterprise-class load-testing tool from Segue. Can simulate thousands of users working with multiple protocols and computing environments. Allows prediction of behavior of e-business environment before it is deployed, regardless of size and complexity. SilkPerformer Lite version also available for up to 100 simulated users.

* Radview's WebLoad - Load testing tool from Radview Software, also available as part of their TestView web testing suite. Capabilities include over 75 Performance Metrics; can view global or detailed account of transaction successes/failures on individual Virtual Client level, assisting in capturing intermittent errors; allows comparing of running test vs. past test metrics. Test scripting via visual tool or Javascript. Wizard for automating non-GUI-based services testing; DoS security testing.

* Loadrunner - Mercury's load/stress testing tool for web and other applications; supports a wide variety of application environments, platforms, and databases. Large suite of network/app/server monitors to enable performance measurement of each tier/server/component and tracing of bottlenecks. Integrates with other Mercury testing and monitoring produces.

Friday, December 22, 2006

News Feeds and Thunderbird

To be able to check news feeds automatically you need RSS reader, the best one to use IMHO is Thunderbird.
Get Thunderbird!

Thursday, December 21, 2006

Sending Subversion Commit Information by Email

On each commit, I want to send email with the commit details to a list of emails.
This is necessary to keep everyone involved updated. We are using subversion version
control system on windows, we use cygwin extensively to automate frequent tasks.
Here are the steps to do so:

- Windows XP Service Pack 2
- Cygwin DLL release version is 1.5.23-2
- email version 2.5.0
- subversion 1.0.6

Sending email:
- The email.exe tool is a nice small tool to send email using sendmail or using other
SMTP server, I use it to send through our company SMTP server.
- First install cygwin, and you must select additional email tool from the email group.
After installation you will have a shortcut on your Desktop.

After download and installation are OK, you need to configure the email tool before using it.
>From now on, I will put $ before each command line, simply this is the bash prompt. You get bash prompt
by running cygwin shortcut on your PC Desktop.

$ email-config
You will be prompted with a series of questions, answer it to setup the email configuration file.

Please enter your From: name (e.g., John Doe) [Your Real Name]:
Please enter your From: email address []:
Please enter your Reply-to: email address []:
Enter 1 to use sendmail or 2 to use a SMTP server [1]:
Please enter the address of your SMTP server []:
Please enter the SMTP port number [25]:
Please select SMTP authentication (1 for none, 2 for LOGIN, 3 for PLAIN) [1]:
Please enter your SMTP username []:
Please enter your SMTP password []:


You can look at /etc/email/email.conf to review your selections and update them as needed.

You can have a look at email man pages by typing, it will show you all the switched used by email tool.
$man email

you can test by sending email to yourself:
$echo "This is a test email body message" > email.txt
$email -s "subject test 1" < email.txt

- The first line create file email.txt with the string "This is a test email body message", of course you can use your favorite editor.
- The second line will just send the email taking the email body from the input stream through "<email.txt"

Now we need to catch svn commit to generate helpful message about the commit and send to email list.

Edit the file c:\svnrepos\myproject\hook\post-commit.bat using your favorite editor, I assume you locate your repository on c:\svnrepos\myproject\

Add the following lines to it:

SET PATH=%PATH%;c:\cygwin\bin\;C:\subversion\bin\;
svn log -v -r "%1" svn://rd-cdma1/myproject > email.txt
svnlook diff "c:\svnrepos\myproject" -r "%1" >> email.txt
email -s "Subversion - %2 - %1", < email.txt

- The PATH environement variable must be set to your programms pathes, I assume c:\cygwin\bin\;C:\subversion\bin\; You must use the correct values that match your system.
- The post-commit.bat will be called every time someone committed new code, this is per subversion implementation.
- %1 is the repository name, automatically passed to the post-commit.bat from subversion.
- %2 is the revision number, automatically passed to the post-commit.bat from subversion.
- > is the redirection operator used to create and append the output to a file.
- >> is the redirection operator used to append at the end of file.
- We use hard coded repository path and URL as the passed parameter %1 has backslashes which doesn't work on Windows.

Using command line tools is very helpful in automating tasks, many of the tasks done by admins requires automation, although GUI is very easy to use at first, you will continue doing things manually. command line give you powerful ways to automate new tasks without developing code which takes much more time.

- Get cygwin from
- Get subversion from

Wednesday, December 20, 2006

Is it time for research?!

I am working now at a company in CDMA terminals, to improve my
understanding I am planning to research CDMA networks, if you know
anyone who can get benefit from such research just let me know !

Tuesday, December 19, 2006

Build you own Linux distribution

It is amazing how it is easy to build your own Linux distribution, look at:

You can also easily build you own Live CD from any working Linux installation, look at:

It is a matter of more several years and Linux will be the public operating system for everyone.

Monday, December 18, 2006

Bash Script Tutorial and Advanced Guide

In automating tasks, we should create scripts, the most common scripting
language available is bash, in my effort to develop personal backup
shell script I need to have bash tutorial, I found "A quick guide to
writing scripts using the bash shell" at very helpful, as
it focus on the basic concepts.

I also made a google search on an advanced guide, and found: "Advanced
Bash-Scripting Guide" at
the pdf version is available at

I hope I can attach the pdf files, but the blog through email feature
doesn't support attachments.

Sunday, December 17, 2006

German Company in Egypt and Open Source Solutions!

I have a friend working as a sales manager in a German company branch in
Egypt, it seems he is a bout to leave them. The company sells Learning
Management Solutions and Business Process Management Solution, look at
these links:

I always prefer open source solutions, this is a strategic direction for me, I found many open source process management software and open source Learning Management System.

Open Source Process Software Resources:
*JBoss jBPM at
*Enhydra Shark Engine
*Enhydra graphical XPDL Java Workflow Editor
*The Workflow Management Coalition at

Open source Learning Management System:

Saturday, December 16, 2006

Java and Arabic Support

Java uses Unicode as native encoding, so any text will be converted to Unicode for proper handling. Java already has support almost to all known encodings, see:

Our involvement is how to adjust the input and the output; the Input will be from request parameters (in case of web development), files, Properties, and JDBC. The output will be to the browser through the HttpServletResponse object or a file, ...

Converting text strings:

  • String class already has support to conversion to Unicode; see String constructors that take encoding as a parameter.
  • String class can convert to any encoding; see getBytes() function that take encoding parameter.
  • String class content at anytime must be Unicode, so it can convert non-Unicode input to Unicode or give you the non-Unicode bytes upon request by getBytes().
  • A running examples is available at
  • You can also use Charset, CharsetDecoder and CharsetEncoder.

Accessing files: (Input/Output)

You can access files, using normal Java classes but be aware that if you did not specify certain encoding, the file input classes will read the system property file.encoding and convert the file content to Unicode based on it.

To know the system file encoding, read System.getProperty("file.encoding")

So if you are writing I18n (Internationalization) applications, you should specify the encoding when you are reading or writing to files:

Use InputStreamReader and InputStreamWriter to specify encoding wanted, see Character and Byte Streams at:

Request Input Arabic Parameters:

If you want to pass Arabic text in URLs, you will get the Arabic as the default system encoding, Cp1252 in a Unicode string, you should convert the parameter back to Cp1252, then to Unicode as an Cp1256, see the example:

String aParUnicode = request.getParameter("apar");

byte [] by1252 = aParUnicode.getBytes("Cp1252");

aParUnicode = new String(by1252, "Cp1256");

By the introduction of Servlet 2.3 you can set the encoding of the request and get the parameters as Unicode correctly.


Response and Locale Object:

Each running application already have Locale object, server applications should specify Locale that matched its language, this will give support in automatic conversion to strings based on this locale, please look at Locale class. The ServletHttpResponse already have a member to set locale called, setLocale

By default Locale is set to en (English) even if your default Windows language is Arabic, so all Unicode will be converted to 8859_1 (Latin1). When you set Locale to ar (Arabic), all Unicode will be converted to 8859_6 (ISO Latin/Arabic Alphabet) and will be displayed correctly as Arabic (ISO).

ISO8859_6 is better than Cp1256, as its compatible with Unix/Mac and Windows, not just Windows.

Properties class:

Properties class are hard coded to read files that encoded as 8859_1 any other characters should be written as Unicode escape character sequence. You can still write Arabic in key values, by it will be converted to Unicode as an 8859_1, so you should convert the Unicode back to 8859_1 and convert to Unicode as Cp1256 or whatever the file real encoding.

String Arabic1256 = new String(latin8859_1.getBytes("8859_1"), "Cp1256");

The above example convert the string from Unicode back to 9959_1 and then back to Cp1256, and assumes the properties file was written using ANSI Cp1256.

  • An alternative solution is to use NetBeans IDE to write Arabic in keys and the NetBeans automatically will convert it to Unicode escape character sequences back and forth, so you always see it Arabic and always stored in the properties file as an escape Unicode character sequences.


Once you have the characters as String Unicode , its already converted to Unicode, so you can convert back to the original encoding and convert to Unicode correctly, some JDBC drivers give you the option to set the charset for the JDBC to make proper conversion like MySQL, the SQL server driver convert to Unicode correctly based on the dbase encoding as selected when you create the dbase.

Be careful if you are using JDBC-ODBC bridge with Access dbase, the driver usually failed to convert Arabic characters to Unicode if your default Windows language is not Arabic.


Java Tutorial Internationalization:

Good ole' ASCII :

The ISO 8859 Alphabet Soup:

J2sdk1.4.0 documentationa and API doc:

>Originally written in 22/5/2002

Friday, December 15, 2006

Conflict, is it that bad?

Conflict is a part of life. There are conflicts of needs, wants, preferences, interests, opinions, beliefs and values. Conflicts are rooted in the way we are created: the diversity of human beings… So conflict is not avoidable.

Conflict is not necessarily bad. Conflicts can be productive, creating deeper understanding and respect, or they can be destructive and harmful. How the conflicts get resolved is the critical factor in determining whether a relationship will be healthy or unhealthy.

Actually conflict is necessary to get high performance; the absence of conflict is a strong indication of a problem and will degenerate into low performance. Fig.1 shows the relation between conflict and performance. The figure shows that too many conflicts and few conflicts decrease performance severely, but moderate amount of conflicts increases performance significantly.

Fig. 1

Sometimes we intentionally create conflict for better performance, as example, the conflict between software developers and software quality assurance. While the development team focuses on finishing the project and closing issues, the quality team is focusing on finding defects in the processes and products. The success of the quality team is to find the largest number of defects which conflicts with the project manager and programmers task as they are focusing on closing issues and finishing the product. This is a very good example of creating conflict for better performance.

In teams, conflict almost exists at all levels, as example between the customer and the software development team, between the project manager and quality manager, between testers and programmers, between employer and the employees, between investor and the community, and between sales and development. These conflicts are not necessarily bad. The team should manage these conflicts for better understanding and agreements to let everyone win. Following is a step for healthy conflict resolution.

Healthy Conflict Resolution:

  • Identify the problem or issues: Have a discussion to understand both views of the problem, conflicts, needs and preferred outcomes. Clarify to each other exactly what the conflict or problem involves. Here are some hints:
  • Listening and not interrupting in order to be able to hear one another's concerns.
  • Summarizing what you have heard to be sure that concerns have been clearly conveyed.
  • Focusing on the problem and not the person. Maintaining respect for the individual while acknowledging disparate points of view will open the door to a considered resolution.
  • Use "I…" statements: This helps you be perceived as expressing yourself so that you are heard and understood, and makes other person more aware of your needs
  • Avoid "YOU..." statements: They come across as attacking which creates defensiveness and counter attack and make the other person more focused on protecting themselves.
  • Generate several possible solutions: Use creative and integrative approach, both parties think creatively about several possible solutions without evaluating them. List all possible solution weather one of the parties agrees on or not.
  • Evaluate the alternative solutions: Narrow down the list to the solution that is accepted to both of you and meets the project objectives and the organization policy and other constraints.
  • Consensus on the best solution: Consensus is the keyword here, both parties try to agree on a solution, if there's no way to agree on a solution, the issue can be escalated to the next level of management for judgment.
  • Follow-up: Follow up to be sure the application of the solution produces the expected results, in case of deviation, changes can be made to the solution, but sure any changes should also be agreed on or escalated.

As a conclusion, conflict is not necessarily bad, moderate conflict and healthy conflict resolution is important to achieve the highest possible performance and quality.

Thursday, December 14, 2006

Arabic Morphological Analysis

Historically - at my former employer Sakhr and Harf - I believed you have to support Arabic morphological analysis in order to have a good Arabic Full Text Search Engine. Recently I was developing an Arabic application using Java, we used Apache Lucene search component, as you know Java already internationalized, so Lucene supports Arabic without much effort. We studied to improve the search by adding morphological analysis.
What it means from the user perspective to have morphological analysis in search, it means searching by كتب (write) and you get results that has many variations, like كتابة، سيكتب، فكتب (writing, wrote, books, ...) and so on. What a stupid idea, Look at Google, if you search by the keyword "reading", it will find it only. Why we should slow down our search algorithm and just adding confusion to the user, Google even search in misspelled words and just offer you to search using the coorect words. What it means?, it means we should respect user input and use it as it, the search engine efforts should concentrate on getting the most relevant hits to the user query at the top.

My conclusion is not the morphological analysis is useless, I believe it is important in machine translation and many other nutural language processing applications but not search engines. The key feature of search engine are accuracy, ranking and of course speed.

If you want to understand Arabic Morphlogical Analysis, I suggest to look at Buckwalter efforts, you can also download his GPL version to learn how things is going on. His code is written in Perl, but you can easily understand his nice readme file, thanks BuckWalter [1] [2].

You can also develop a shallow morphological analyzer using the techniques suggested by Kareem M. Darwish, you can look at his web site [3], he has a nice research paper called titles "Building a Shallow Arabic Morphological Analyzer in One Day" [4]. A free software tools are available for free.

Research in Arabic morphology analyzer increased nowdays, you can find amazing number of research papers and dissertation online at Google Scholar [5].

Note: Later Buckwalter ported the code to java with GNU licenese, look at:



* This post originally written on Jan 16, 2006

Wednesday, December 13, 2006

PSP, the CMM for the Single Programmer

In this short article, I will try to give a simple overview of the PSP as we practice it, what I am describing here reflects my understanding and my practice. If you want formal articles that describe what PSP is, please refer to the references section in this article.

PSP is CMM level 5 applied to a single programmer. In other words, how a single programmer could apply engineering discipline on just himself. I really like this approach, as it comes to basics, the single programmer who is writing source code for the system.

Many small corporations in Egypt have small projects that a single programmer develops from start to end, such projects exist even in medium corporations, especially if they are using modern high level languages, modern development tools, and their large reusable code libraries. For the above reasons, I feel happy with the PSP training, as we are again going to basics and speaking practically; this could help us in our corporation to develop better.

In a nutshell, PSP is a way to develop software using a defined process. The process describes how to collect quantitive metrics regarding performance and quality; and how to find ways to improve continuously.
The magic of PSP formal training is that, we practice it through 10 real programming assignments; some of them are not trivial. So, the PSP training can't be taken by non programmers.

The relation between PSP and CMM:

In Fig-1, we show the CMM levels and mark by "*" all key practices that at least partially are covered by PSP. It's clear that a large percent of CMM key practices are covered in PSP.

Fig-1 (from "A Discipline for Software Engineering by Watts S. Humphery")

PSP Levels:
As everything in life is developing incrementally, so is the PSP: We started with PSP0, PSP1.0, PSP2.0 and then PSP3. Every new process version introduces new changes in the process. This incremental approach is really natural; I don't expect any success if someone is going to practice PSP2 or PSP3 directly.

In Fig-2, we show the levels of PSP, in each level, new concepts are introduced and a new programming assignment is given to practice the new concepts in actual programming work.

Fig-2 (from "A Discipline for Software Engineering by Watts S. Humphery")

PSP0: The Baseline Process:
The baseline is simply what we already do in developing software, getting requirements, creating design, writing code, and testing; but you would record all the time spent in each phase and would record all the defects you found and how much time it took to fix them.

In the Postmortem report (a report you write at the end of every assignment), you count how much defects are injected and removed from each development phase. Actually PSP0 is the basic training on using a defined process and writing basic reports.

We started the first assignment using paper forms which were tedious but helped us to concentrate on the new concepts rather than to use a tool that could distract us from the new concepts. In later assignments, we used a nice spreadsheet that is provided by the instructor.

In PSP0.1 we start the basis to measure size by following a coding standard. We also used the PIP (Process Improvements Proposal) which is a structured document to record our process problems and improvements suggestions.

PSP1.0: The Personal Planning Process:
The size estimation OLOC (Object Lines Of Code) is used and compared to the actual. We use a size template to write all objects (Classes in C++ and Java) and how much functions will exist in each object and the expected lines of code per function.

Now with a history you can just estimate OLOC and let the spreadsheet calculate the expected time by using statistical linear regression once there was a correlation [linear relation] between your old LOC and time in previous projects. Of course the estimation will be based on your past data, so you should make sure to record consistent and accurate data to get quality estimates. The spread sheet will also compute automatically the time spent in each development phase based on your past history.

This was a great step, I don't have to estimate time anymore, I will just estimate size using familiar Object Oriented techniques and then estimate the lines of code. The time is automatically calculated using statistical linear regression.

Also in this phase we learn to write test report to record all test cases used to verify the program is correct.
In projects that span many days or weeks, task planning and schedule planning will be a critical issue. First you estimate size and time required (task planning), then make a schedule. The schedule will allocate already estimated tasks to real resources. The schedule tracking is so important, and a corrective action is necessary once a plan slip is detected.

PSP2: Personal Quality Management:
At this step, we already have a very good data about our defects, we analyze defects carefully and devise several approaches to minimize them:


Using structured design review and code review that is based on a checklist, we can greatly reduce defects even before the first compilation and test.
In order to improve the review effectiveness, a peer review is introduced. As we know, we get used of our mistakes, so that we no longer see them. Other minds/eyes will easily detect many of these defects.

Design process:

Through the focus on how to verify the completeness of a design, not necessarily how the design it self is made, verifying the design completeness will eliminate a large source of defects. Actually design completeness check can be done on many phases, like requirements; sure it will be ideal to have requirements completeness criteria before development.

PSP3: A Cyclic Personal Process:
Using PSP2 is perfect in small scale projects, but what if you have a large project? The idea is to divide the project into many PSP2 projects, so each cycle of development is based on high quality previous cycles. If the previous PSP2 cycles are badly done, the test will be a complex task, so we focus on doing PSP2 completely in each cycle.

Tool Support, the PSP Dashboard:
Of course tool support will help us to make following the process easier, I tried to use process dashboard as advised by our instructor Dan Roy, and find it helpful. I encourage every trained PSP to use it, however I think it will be complex and unintelligible to anyone who is not familiar with the PSP through formal training or at least with the help of a formal PSP programmer.
Finally, we started to use PSP in-house, but we still have no concrete experience until now, I hope that anyone who would have that experience to share it with us.

References and additional resources:

Tuesday, December 12, 2006

Monday, December 11, 2006

Browsealoud... Making Websites Talk

Browsealoud reads web pages aloud for people who find it difficult to read online. Reading large amounts of text on screen can be difficult for those with literacy and visual impairments.

Sunday, December 10, 2006

HTML Validator based on Tidy that Supports Arabic

HTML Validator is a Mozilla extension to FireFox that adds HTML validation inside Firefox and Mozilla.
It supports Arabic text without any problem, it corrects the tagging and you can get the correct page easily.

See it at:

See FireFox at

Saturday, December 09, 2006

NetBeans Remote Debugging

This tutorial will show how to debug Java code on remote Tomcat process, this steps are tested on Windows XP,JDK 1.5 and Tomcat 4.1, I believe it will work also on Linux and recent version of Tomcat.

It's actually beneficial to debug your applications on a separate Tomcat instance instead of the Tomcat built in Netbeans, this especially important in complex and large Java projects to help reduce complexities. Generally you can remotely debug from any Java IDE into any remote servlet server process, in this tutorial I am just showing how to do it between Netbeans and Tomcat.

First, the Tomcat process must run in debug mode, this means you can't use the backgroud service in Windows, you have to use batch files.

You can use this command line to start Tomcat in debug mode: (after adjusting folders directories)

C:\jdk1.5.0_02\bin\java.exe -jar -Xdebug -Djava.compiler=NONE -Xrunjdwp:transport=dt_socket,server=y,suspend=n,address=7001 -Duser.dir="C:\Tomcat4.1" -Djava.endorsed.dirs="C:\Tomcat4.1\common\endorsed" "C:\Tomcat4.1\bin\bootstrap.jar" start

As I said it is a ommand line, so it's all one line.

You can also run Tmcat in debug mode using the batch files already distributed with Tomcat.
  1. Under Tomcat\bin make a copy from "startup.bat" and name it "startup_debug.bat"
  2. Make copy from "catalina.bat" amd rename it to be "catalina_debug.bat".
  3. modify "startup_debug.bat", make it call "catalina_debug.bat" instead of "catalina.bat".
  4. Modify "catalina_debug.bat" by adding -Xdebug switch:
CATALINA_OPTS=%CATALINA_OPTS% -Xdebug -Djava.compiler=NONE -Xrunjdwp:transport=dt_socket,server=y,suspend=n,address=7001
  1. Saved it and you have finished.

Now its the step of attaching to Tomcat from Netbeans,

in Netbeans:
Run the followng menu commans:
Debug -> Start Session -> Attach -> Connector: Socket Attach -> Host:
"localhost" and Port: 7001

Set your breakpoints and enjoy it.

Friday, December 08, 2006

The State of "Food Insecurity in the world 2006", a Report

The State of "Food Insecurity in the world 2006"
Download Poster, Flayer and Full Report from
>From the report:
854 million people – one in seven of the world's population – lack access to sufficient food.
In some countries, more than one third of the population suffer from hunger.

According to Hartwig de Haen, Assistant Director-General, Economic and Social Department, more than 25,000 people die of starvation every day, and more than 800 million people are chronically undernourished. On average, every five seconds a child dies from starvation.
See this article at

Wikipedia has a good article at

You can see hungry zones at:

You can see also the progress of hungry zones over years on this flash page:

Undernourished people (as % of total population):

Here are some countries percentage:
Egypt 4%, Somalia 71%, Sudan 21%, Yemen 33%, Afghanistan 70%, Bangladesh 35%,
Pakistan 19%, and middle Africa countries around 40%

Thursday, December 07, 2006

The 7 Habits of Highly Effective People

Habit 1: Be Proactive
Change starts from within, and highly effective people make the decision to improve their lives through the things that they can influence rather than by simply reacting to external forces.

Habit 2: Begin with the End in Mind
Develop a principle-centered personal mission statement. Extend the mission statement into long-term goals based on personal principles.

Habit 3: Put First Things First
Spend time doing what fits into your personal mission, observing the proper balance between production and building production capacity. Identify the key roles that you take on in life, and make time for each of them.

Habit 4: Think Win/Win
Seek agreements and relationships that are mutually beneficial. In cases where a "win/win" deal cannot be achieved, accept the fact that agreeing to make "no deal" may be the best alternative. In developing an organizational culture, be sure to reward win/win behavior among employees and avoid inadvertently rewarding win/lose behavior.

Habit 5: Seek First to Understand, Then to Be Understood
First seek to understand the other person, and only then try to be understood. Stephen Covey presents this habit as the most important principle of interpersonal relations. Effective listening is not simply echoing what the other person has said through the lens of one's own experience. Rather, it is putting oneself in the perspective of the other person, listening empathically for both feeling and meaning.

Habit 6: Synergize
Through trustful communication, find ways to leverage individual differences to create a whole that is greater than the sum of the parts. Through mutual trust and understanding, one often can solve conflicts and find a better solution than would have been obtained through either person's own solution.

Habit 7: Sharpen the Saw
Take time out from production to build production capacity through personal renewal of the physical, mental, social/emotional, and spiritual dimensions. Maintain a balance among these dimensions.


You can look at a longer summary at:

The summary can't be sufficient without reading the book, buy a copy from

Wednesday, December 06, 2006

Software Version Control - Subversion- Installation and Configuration

>> There is a bug in Microsoft Internet Explorer 6.0, this article is best viewed with FireFox 2.0

Software Configuration Management (SCM) in Roger Pressman book Software Engineering: A Practitioner's Approach, says that software configuration manage ment (SCM) is a "set of activities designed to control change by identifying the work products that are likely to change, establishing relationships among them, defining mechanisms for managing different versions of these work products, controlling the changes imposed, and auditing and reporting on the changes made.". In other words, SCM is a methodology to control and manage a software development project.

SCM concerns itself with answering the question: somebody did something, how can one reproduce it? Often the problem involves not reproducing "it" identically, but with controlled, incremental changes. Answering the question will thus become a matter of comparing different results and of analyzing their differences. Traditional CM typically focused on controlled creation of relatively simple products. Nowadays, implementators of SCM face the challenge of dealing with relatively minor

increments under their own control, in the context of the complex system being developed.

A corner stone tool to start with is the revision control system, so in this article we will focus on it and explain its installation and configuration.

Revision control - also known as version control - is the management of multiple revisions of the same unit of information. It is used to manage ongoing development of digital documents like application source code, art resources or design and architecture do

cuments. Changes to these documents are identified by incrementing an associated number, termed the "revision number" and associated historically with the person making the change. A simple form of revision control, for example, has the initial issue of a drawing assigned the revision number "1". When the first change is made, the revision number is incremented to "2" and so on.

Software tools for revision control are increasingly recognized as being necessary for software development projects, not to manage only source code, but also for all software artifacts.

We introduce one of the best revision control system, Subversion, and will make guidance on how to install, configure and use effectively. In general use subversion –SVN – to control any document, which is text based or even binary files, in the case of text files you will be able to use tools to show revisions differences.

Subversion Basic Features:

  • Open source. Being open source does not mean low quality; it is a commercial quality system.
  • Multiplatform, Windows, Linux, and many other.
  • Commits are truly atomic operations. Interrupted commit operations do not cause repository inconsistency or corruption.
  • Renamed/copied/moved/removed files retain full revision history.
  • Directories, renames, and file metadata are versioned. Entire directory trees can be moved around and/or copied very quickly, and retain full revision history.
  • Native support for binary files, with space-efficient binary-diff storage.
  • Apache HTTP server as network server, WebDAV/DeltaV for protocol. There is also an independent server process that uses a custom protocol over TCP/IP.
  • Branching and tagging are cheap (constant time) operations.
  • Natively client/server.
  • Client/server protocol sends diffs in both directions, so costs are proportional to change size, not data size.
  • File locking for unmergeable files ("reserved checkouts").

We will use RapidSVN as a multiplatform Subversion client tool, see Fig 1, keep in mind that this is a personal choice, more than 20 client tool and IDE plug-in are available at

Fig 1

In order to help you introduce Subversion in your work, I will show steps of installation, configuration and basic use, I will cover only Windows-based installation.

Download and Install:

  1. Download subversion client/server software from
  2. Install the downloaded file to c:\subversion
  3. Add c:\subversion\bin to the Windows PATH variable.
  4. Download RapidSVN from
  5. Install RapidSVN to any place in your hard disk, and keep its icon reachable.
  6. Run RapidSVN and explore it a little to be familiar with.

Configure and Use:

  1. We simply create a repository per project.
  2. Create a root folder for all your repositories, create c:\svnrepos\
  3. Create initial repository:
    Open command shell, and run
    svnadmin create c:\svnrepos\project1
  4. Open Command prompt, run the server by running the command
    svnserve -d -r c:\svnrepos
    this means you want to run the svnserve as a daemon to serve repositories under c:\svnrepos.
  5. open svnserve.conf under c:\svnrepos\project1\conf , remove the comment from the following lines:
    auth-access = write
    password-db = passwd
  6. Open passwd under c:\svnrepos\project1\conf, uncomment the following lines:

    add users in the format that user=password, as shown above.
    There exist many options in authorization and authentication, see references section for additional documents.

Check out a version from the repository:

1. Run RapidSVN.

2. Press Ctrl->O write svn:// in URL field, (I assume the server machine IP is, write Destination Directory, I assume c:\projects\project1, then press OK.

Add Files:

1. Create new file, name it file.txt, write the statement This is version 1, save the file, copy it under c:\projects\project1 . Open RapdSVN, open the bookmark added previously, you will see the file status unversioned, right click the file and select Add command. Now the file status is added, but the Rep. Rev. is -1. This means that you still have to commit the file in order to add it to the repository, right click the file and select Commit…, now the file status is empty. And Rep. Rev. is 1.

2. Open the file1.txt in any text editor, add second line as following This is text line number 2. Save the file. In ReapidSVN the Status is modified. To commit the file to the repository, right click and select Commit…, you will be authenticated, write matching user name and password that match one that is created in passwd file, after OK, the Rep. Rev. is now 2.

Text Viewing, Differences and Merges:

  1. For text files viewing, you can use Notepad++ from, you can also use Porgrammers Notepad at Both are open source and good quality.
  2. For text files differences and merge tool, WinMerge is an excellent tool, download from
  3. In RapidSVN, select View > Preferences and select Programs tab, adjust the path of Standard Editor and Diff Tool as shown in Fig 2.

Fig 2

The following guidelines are our practical experience that helped us to use subversion properly in a way that helps us make our source code versions manageable.

  1. Update frequently as much as you can, commit on average daily. In case of any conflict consult software team leader and other team members, in 95% of cases you can resolve the conflict safely without any guidance.
  2. Commit each logical unit of change separately, as example if you are going to fix three bugs, commit each bug separately, this will ease changes tracking.
  3. On every commit, write detailed notes.
  4. Don't ever commit code that breaks the build.
  5. Don't change any code that you don't own without consultation to the code owner.
  6. Before committing run self-test-procedure for your changes.
  7. Use WinMerge always to review changes carefully before each commit.
  8. Don't put generated files into version control, put only source files whatever it is textual or binary files like images or word document.