When was last time you spent waiting more than 2 seconds for a page to load? The average user has no patience to wait too long for a page to load. Users lose their interest in a site if they don’t get a response quickly, people like fast responding websites. The Riverbed Global Application Performance Survey has revealed a major performance gap between the needs of business and its current ability to deliver.

• According to the survey 98% of executives agree that optimal enterprise application performance is essential to achieving optimal business performance.
• 89% of executives say poor performance of enterprise applications has negatively impacted their work.
• 58% of executives specified a weekly impact on their work

Poor app performance impacts every area of the business.

• 41% cited dissatisfied clients or customers
• 40% experienced contract delays
• 35% missed a critical deadline
• 33% lost clients or customers
• 32% suffered negative impact on brand
• 29% faced decreased employee morale

Application performance should be on top priority; every millisecond matters, few milliseconds difference is enough to send users away.  Performance optimization saves time, money and valuable resources. Our team was assigned on a critical mission to bring down a core business API execution time from 21 seconds to 4 second. I am sharing my experience on this mission; hopefully it will help you in understanding performance monitoring process and optimization techniques. Performance improvement is an ongoing iterative process. This blog discusses more about server side application tuning. On reading this article you would learn

• How to initiate application performance tuning
• Performance monitoring
• Identifying optimization areas and optimization techniques

Below is the sequence of steps in performance optimization.

• Benchmarking
• Running performance test
• Establish a Baseline
• Identify bottlenecks
• Optimization

Below diagram shows typical process of running performance initiative Benchmarking Benchmarking can be simply defined as “setting expectation”. An athlete sets a benchmark of running 200mtrs distance in 20 seconds (athlete is setting expectation here), similarly a product owner sets a benchmark for a Login API that it needs to be executed not more than 1000ms for 15 parallel users. API will be running on 3 application servers under load balancer having 1TB External memory, 12GB RAM, running on Intel i7 core processor each. Application server connecting to DB server will have same hardware configuration as an application server. These are examples of benchmarking. It’s very important that system Hardware Configuration (HDD, RAM, and no. of server), application configuration and acceptable payload must be fixed during benchmarking and ensure it remains unchanged during all subsequent performance tests. Performance environment generally are replica of production environments. Running Performance Test  The purpose of the performance testing is to measure the response time of an application/API with an expected number of users under moderate load. It’s generally done to establish a baseline for future testing and/or measure the saving over baseline on performance related code changes. Performance tests can be carried out by using tools like SOAP UI, Load Runner, etc. Ensure you’re having same configuration, payload as fixed on benchmarking before running test. You also need to have a performance monitoring tools like AppDynamics, ANTS profiler, etc. configured to capture call graph for the executed application/API. These tools help in analyzing and identifying bottlenecks. Establish a Baseline Baseline is a system assessment that tells how far we are from benchmark figures; Current state of the system is a baseline.  It is an iterative process; it keeps evolving on code changes. Currently athlete runs 200mtrs in 25 seconds is an example of baseline (Athlete is still behind by 5 seconds from benchmark example above), a test performed on a login API with same criteria used as benchmarking takes 1500ms (still 500ms behind of benchmark figures as mentioned in example above). This gap between benchmark and baseline need to be filled in by increasing performance so that baseline figures are equal or less than benchmark figures.  Identify bottlenecks Increasing performance would first need identifying the bottlenecks. This is very important part of performance tuning, it needs keen observation. Performance monitoring tool gives you report with detail call graph and statement wise time taken. Call graph need to be further analyzed and narrowed down to the root cause of performance issues, ensuring no single opportunity goes unnoticed. Hardware bottlenecks – objective is to monitor hardware resources such as CPU utilization, memory utilization, I/O usage, load balancer, etc. to see if it has some bottleneck. Software bottlenecks – monitor webserver (IIS), DB server, etc. to see if it has any bottleneck. Code bottlenecks – No matter how careful and attentive your developer team is, things are going to happen. Identifying code bottlenecks is a technique, find out code areas that takes more resources or execution time. Finding out such code areas will open more performance opportunities. Below are few common code bottlenecks to be looked for. Identify Methods/block/statement on call graph that takes long time to execute. Find duplicate DB/IO calls on a call graph. Identify long running SQL queries/IO operations. Optimization After finding bottlenecks, next step is to find out the solution on identified bottlenecks. Hardware optimization – if you see high memory or CPU utilization during performance test run, analyze code and find root cause of issue. There are many possible reasons behind this issue. E.g. memory leaks, multi threads, etc. If you particularly find multithreading reason behind high system consumption, try combining these threads statements into main thread to run it synchronously. Obviously, it will increase overall execution time. If you can afford it, go ahead and implement change else execute these threads asynchronously on other server to keep application server health under control without compromising overall execution time. Address any other hardware bottlenecks if found. Software optimization – Analyze bottleneck and find root cause. You may sometime need to involve respective expertise (IIS, Database, etc.) Code optimization –

• If possible, use object cache on heavy non-changing objects.
• Check if time taking statements/methods can be executed asynchronously without violating fair system usage.
• Use proper Indexing on table to increase query performance.
• Check call graph to see if same method getting called multiple of times, if so apply appropriate cache mechanism to avoid these duplicate DB calls.

Conclusion Good design and coding practices leads to high-performance applications. Irrespective of the power of the hardware, application can be inefficient when not designed well and not optimized. Many performance problems are related to application design rather than specific code problems. It’s very important to have high performance applications in this highly competitive market to grow and sustain.  We see many applications failing when data grows significantly, as data grows performance becomes crucial, it’s important to keep application performance consistent even if data grows. At Xoriant, we have specialized team which works on Performance tuning and monitoring that help clients to tune their critical enterprise applications performance even with large data sets.