Friday, May 5, 2017

Logging the right way

Logging is an important aspect of software development, one that is many a time ignored by developers.
it's important to have clean and helpful logs. They can come real handy when you have to debug an issue.

Below are a few important points to keep in mind while writing your logs

1. Using a good API

In my experience,I find the slf4j API better than the others.
Consider this:

in log4j:
logger.debug("Found employee : name: "+name+"designation: "+designation+" department: "+department);
This is difficult to read and has 3 string concatenation.

it can be rewritten with slf4j as
logger.debug("Found employee : name: {} designation:{} department: {}",name, designation, department);
Now, this is more readable, and avoids the string concatenation.

2. Watch what you are logging

Every time you log, watch what you are logging. Sometimes loggers can be the cause of many exceptions

Example 1: logger.debug("User Details :{}", user.getName());
This might throw a null pointer if the user is null

Example 2: logger.debug("User Details :{}", user);
This is an example where the hibernate object has been sent to the logger. This can give way to one or more of these issue :
Out of memory error, N+1 select problem, lazy initialization exception, logs storage completely used up

It' always a good idea only to log the ids of domain objects instead of the complete object. They are enough information for debugging.
Log no more than necessary in release/production, excessive logging can make you app slow.

The following logger format works best for me:
Who(UserName) , When (Timestamp), Where(Context, ServletOrPage,Database), What (Command), Result (Exception/Error)

3. Use categories

The severity values INFO, WARN, ERROR, and DEBUG, can be used to filter logs based on the severity.
The benefit that loggers have over System out is the logging categories and levels.
To read more about logging severity, visit

Logging is an I/O intensive activity and can get in the way of the application's performance, particularly when logging is done in the traditional way that is, synchronously writing to a FileAppender in the context of application threads.
It can often contribute to heavy heap consumption and garbage collection.
hence, excessive logging can lead to performance issues. So be concise, and log relevant data that would help in debugging.

You can also use isDebugEnabled() to prevent filter the logs for production.

4. Log files locally
Use a file local to the server to write the loggers. It provides a local buffer and you aren't blocked if the network goes down.

5. Rotation policies
Log files can get huge. Maintain a rotation policy to roll logs periodically. You can also have a strategy to destroy or back up your log files after a period of time.

Thursday, March 9, 2017

The power of Continuous Integration

What is Continuous Integration?

Continuous Integration is a methodology followed by development teams, where in they submit their codes periodically to a source control management system (SVN, ClearCase, Git etc.). Each time a new submission happens, a set of automated scripts run to build, deploy and test the build. This ensures that the current build is error free and a clean deployment.

There are many tools available for Continous Integration...TeamShip, GitLab, Codeship. The most commonly used one is Jenkins.
I will talk about the most commonly used CI tool, Jenkins.


This is just the tip of the iceberg. There are many advantages that CI brings along:

1. Integrations are automated
The integration process can be completely authomated. Configure your Jenkins to poll the repository on a regular interval. Once the tool detects a check-in, it triggers a set of configured scripts to automatically buils and deploy the build to the server. Thsi saves time and also eliminates manual errors

2. Testing can be automated
Jenkins can be used to trigger a set of automated test cases each time a build is triggered. This ensures that the current build does not break any existing use cases.

3. Early detection of issues
CI can help find the issues before they can appear on the deployed application on the server, thus helping finding issues earlier and causing substancially less damage

4. Time Efficient
You save a lot of time lost due to debugging, manual integration and the deliveries can be done faster

5. Reduces manual work
Equates to saving time, money and less errors due to an authomated process

Features provided by Jenkins:

There are a numerous things that are provided by jenkins, among others

1. Addons: Jenkins can be easily extended by using over 400 plugins that are available.

2. Source Control: Jenkins can directly connect to all major version control systems and can poll periodically for changes. It can also check code back in to the version control

3. Job Configuration History Plugin: Tracks the changes to the job configurations, including who did it.

4. Automation: Schedule automatic builds based on cron expressions

5. Testing: Configure automated test cases to be triggerd on each build. This can trigger an automatic smoke test for each build.

6. Shell Scripting: Can add pre and post build operation by using basic shell scripts.

7. Notifications: Notifications can be configured to be sent to a set of people for the build status.

8. Access Control: The jenkins admin has an elaborate dashboard to provide multiple levels of access to various users.

9. Sequence the jobs: The jobs can be sequenced so that the successful completion of one can trigger another job.

10. Distributed builds: The master slave architecture supports distributed builds. You can build on one server and deploy on another. This reduces the load on the servers

11. Publish various reports: The jenkins plugins can be use to publish various reports in graphical or tabular form, e.g test coverage, code coverage and static code analysis reports.

How to make the best use of CI

Here are some best practices so that the CI features can be used to its best:

1. Maintain a single repository.

2. Make daily check-ins. It is a good practice to check in your code daily and not have any uncommitted code in your local system.

3. Automate the CI job to poll the version control periodically so that build are triggered every time a code is checked in. This way its easier to know which commit has failed the build.

4. Have automated test cases to run for each build to ensure nothing breaks due to a new build.

5. Maintain the unit tests, they can be triggered as a part of the build and verify the build.

6. Publish each build to server, so that the latest build is available for everyone to see.

7. Publish notifications to owners for failed builds, so that actions can be taken quickly.

8. Publish relevant reports. Eg: Test coverage, Code coverage, static code analysis reports etc. A good reports generates confidence to the stakeholders.


This requires commitment from individual team members to commit code frequently and ensure that they do not check in broken and untested code. So this would require to educate and empower the team members.
Adopting the Continuous Integration and Continuous Deployment will help catch the issues early in time and hence reduce risks of errors and ensure better quality.