Dev Environment Revamp

image

For long we have been having our traditional environment setup: Dev, Stage & Production. With us adding more and more features into our product and with a large development team of about 50+ developers, this traditional structure became insufficient.

The most issues used to happen in the Dev environment where if one developer wants to test only his code and has put an 'exit' in his code path, it used to affect everyone else. The problem becomes acute when the change is is some commonly used functions/includes.

The impacted team members had to find if it is problem with their code or somewhere else and this killed the team productivity. It usually takes a few minutes to few hours before things become normal. With no way to recuperate lost time, our quality cycle was getting impacted.

And developers need to be given space to do their experiments, so as to do justice to their function. We had to overcome this issue and had 2 choices:

a. Have one more environment created like pre-prod and have developers test

b. Relieve the stress on 'Dev' environment by making the developer workstation as the playground with access to DB/indexes from a separate location.

And obviously we chose the 2nd option. It was pretty frugal too. This setup gives complete freedom to developers to experiment the way they want and not impact anyone else.

With help from our Infra team, we setup a local server accessible for all developers that carried our Mysql tables, our Sphinx & Solr Indexes, MongoDB, Redis & memcache instances. Also in the older setup, we were dependent on our Infra team for DB operations/Index rotations. Since this a completely isolated environment, we took over that function and had 100% freedom which saves time too.

Our product runs on both NodeJs in some modules and PHP in some, with our focus to shift completely to the former in the coming months.

Now we had to install tools & software in each developer machine that makes them code independently. And typically your page might have more than one end point[sub domains] to load all its images, contents, forms. That would also not be a problem.

For PHP, we installed XAMPP. The following configurations changes were to be made in the apache - httpd-vhosts.conf file as

 < VirtualHost *:80 >
##ServerAdmin webmaster@dummy-host.example.com
DocumentRoot "C:/xampp/htdocs/profiletamil/www/"
ServerName profile.tamilmatrimony.com
ServerAlias profile.tamilmatrimony.com
##ErrorLog "logs/dummy-host.example.com-error.log"
##CustomLog "logs/dummy-host.example.com-access.log" common
< / VirtualHost > 

[You can repeat the above code for any number of sub domains you have]

To further improve our code quality, we mandated used to VScode editor instead coding in Notepad or EditPlus. Also we made it mandatory to install the plugins: Xdebug, PHPCS, PHPMD and PHP Formatter (PSR2) to improve our coding standards and quality.

We could write our own rules for maintaining code standards and code quality in PHPMD. We defined the rules in ruleset.xml file for identifying unused code, codesize and cyclomatic complexity.

How we are using Xdebug in our dev environment:

1. Xdebug is a PHP extension which provides debugging and profiling capabilities. It uses the DBGp (debugging protocol).
2. It is used to identify the right spot for optimization. Don't waste the time on wrong spots.
3. PHP developers default flow tracking methods are :
a. echo / var_dump()
b. writing LOG at required places (which will increase the IO on the server)
All these methods require a modification of the source code and come at a performance penalty
4. Xdebug beautifies the var_dump() output to make it even more readable.
6. We can create a trace log with Xdebug which helps us to understand the control of the complex programs.

PHP.ini setting:
--------------------
zend_extension="/usr/lib64/20090626-zts/xdebug.so"
xdebug.remote_enable=1
xdebug.profiler_enable=0
xdebug.profiler_enable_trigger=1
xdebug.remote_handler=dbgp
xdebug.remote_mode=req
xdebug.remote_port=9000
xdebug.profiler_output_name=cachegrind.out.%H.%s.%t
xdebug.trace_enable_trigger=1
xdebug.auto_trace=0
xdebug.trace_format=1
xdebug.trace_options=0
xdebug.show_mem_delta=On
xdebug.trace_output_dir="/var/log/xdebuglog"
xdebug.profiler_output_dir="/var/log/xdebuglog"
xdebug.idekey=netbeans-xdebug
xdebug.remote_log="/tmp/xdebug.log"
xdebug.remote_connect_back=1
xdebug.show_exception_trace=1

The function xdebug_start_trace starts the trace logging, and xdebug_stop_trace stops trace logging. If you want xdebug to show an additional memory delta column in the trace log, add xdebug.show_mem_delta=On. The memory delta is the difference between the memory size in the current line and the previous line.

<?php
xdebug_start_trace('c:/data/fac.xt');
print fac(7);
function fac($x)
{
if (0 == $x) return 1;
return $x * fac($x - 1);
}
xdebug_stop_trace();
?>

We can also configure xdebug to log the parameters that were passed to a function. This is very helpful to better understand the program control flow
xdebug.collect_params=On

we can configure xdebug to log the return values of each function to better under the program flow as
xdebug.collect_return=On

The function tracing feature of xdebug is very useful. You can create logs in your application without adding a single call to a logger function. You should never activate trace logging on a production site, because that would absolutely kill your performance as one line has to be written to a file for every function call in your PHP script.

Profiling with Xdebug :

Basically, profiling is about counting how many times a statement has been executed and how much execution time it took. Profiling an application is the first step in optimizing it, because you need to know where your application spends most time, before you can figure out how to make your application faster.
1. Creating a profiling log seriously degrades your PHP script performance, like creating a trace log does, because several lines have to be written to a file for each function or method call. Therefore, you should never profile on a production system..
2. Xdebug writes profiling logs to /tmp by default. To change the default dir use xdebug.profiler_output_dir="/usr/local/bin/xdebug_log".
3. Sometimes, you don't want to profile all files, instead of permanently turning on the profiler, you can add xdebug.profiler_enable_trigger=On to php.ini. Now you can enable and disable profiling by passing the special GET or POST parameter XDEBUG_PROFILE to a PHP script. This will turn on profiling just for the one PHP script that receives the parameter in the URL as test.php?XDEBUG_PROFILE..
4. The tool commonly used with Xdebug profiling data is KCachegrind. KCachegrind is freely available under GPL, but runs only on Unix. Luckily, there is a similar tool for Windows, WinCachegrind, which is also free software..
KCachegrind visualizes the profiling data and draws call graphs.

Creating Code Coverage Statistics with xdebug :

1. Code coverage statistics show how many times each line of the code has been executed. Conversely, they also show which lines of code have not been executed, which is in fact much more interesting. These parts are either obsolete or should be tested.
With the help of PHPUnit xdebug will do this code coverage statistics

More stories on code quality is coming soon...