Continuous Testing of PHP Compiled to .NET Core on Linux – Technical Report

In the previous blog we have explained all the benefits of the continuous testing on Travis CI. Now, we want to show the way it is implemented from a more technical standpoint.


After a change of the repository or a pull request, Travis CI automatically starts a disposable virtual machine and runs the commands we define. To summarize, we want these actions to happen:

  1. Install all dependencies
  2. Compile Peachpie to .NET Core
  3. Test that Peachpie is able to compile a sample PHP application
  4. Compile every PHP test file to .NET Core, run it and compare the output with the PHP interpreter

Compilation in Travis CI

In order to achieve this in Travis CI, we need to create the .travis.yml configuration file:

sudo: required
dist: trusty
language: php
  - '7.0'

  - ./build/
  - dotnet restore ./src
  - dotnet build -f .netcoreapp1.0 ./src/Peachpie.Compiler.Tools
  - dotnet pack --no-build -c Debug -o ./.nugs --version-suffix dev ./src/Peachpie.Compiler.Tools/project.json
  - rm -rf ~/.nuget/packages/Peachpie.Compiler.Tools
  - dotnet restore ./Samples
  - dotnet run -f .netcoreapp1.0 -p ./Samples/ConsoleApplication1
  - ./src/Tools/runtests_shell/

In the upper part, we specify the type of virtual server and its parameters. Particularly, we demand the one with pre-installed Ubuntu 14.04 LTS (Trusty) and PHP 7.0. We also state that root privileges are needed so that we can install the dependencies. That is done in the first command of the install section, namely in the script. The following libraries and tools are installed:

  • .NET Core - to build and execute our projects
  • Powershell – to allow us to write cross-platform scripts
  • icdiff – to pretty-print the differences of output in failed tests

The first one mentioned is particularly important, because it enables us to build and execute .NET applications on Linux. See the official installation instructions for more information.

Afterwards, we run dotnet restore on all our C# projects so that all the necessary packages are installed and all the dependencies between the projects are solved. Next, we can proceed to the compilation of Peachpie. The project Peachpie.Compiler.Tools, which is the compiler implementation for .NET Core, contains a post-build Powershell script named update_packages.ps1. It turns the compiled projects into NuGet packages, stores them into the .nugs directory and installs them.

There is a minor problem caused by packaging the application in the post-build step – after the first build, the package does not contain all the necessary files. This is solved by the commands on lines 11 and 12, which recreate the package and remove the installed one.

In order to build our sample projects using Peachpie within the .NET Core build system, several customizations of their project.json files had to take place. For more information, refer to the corresponding tutorial on our blog. Among others, the Peachpie.Compiler.Tools package was added as their build-time dependency. Running dotnet restore on our samples therefore installs this package and its dependencies from the .nugs folder to the package storage of the user. As the contained command has a special name (dotnet-compile-php), .NET Core is now able to build projects written in PHP. Calling dotnet run on the ConsoleApplication1 sample will build its only PHP file using Peachpie into a .NET Core application and launch it. As a result, Hello World! appears in the Travis CI console and we are safe to continue to the tests itself.

Running Tests

The basic idea of the testing is pretty straightforward: having a collection of PHP files in the tests folder, we compile each of them, compare the output with the one from the standard PHP interpreter and display the results. The problems related to these tasks are explained below, the final solution can be found in

Compiling Multiple Files

To compile a PHP file using Peachpie in the .NET Core build system, it needs to be a part of a project. When we issue a command to build it, the corresponding project.json file is inspected and several helper files are created before passing control to Peachpie.Compiler.Tools. This approach would obviously be cumbersome, because we would need to have a project.json for every tested file. Therefore, we decided to create the helper files ourselves (see the test tool directory) and then call Peachpie.Compiler.Tools directly.

Displaying Results

In Travis CI, the only direct output channel is the console log, so we cannot simply use any fancy HTML5 formatting of the test results. However, at least text folding and coloring is enabled there, so some nice results can still be achieved. The folding automatically applies to all the successfuly executed commands from the install section of .travis.yml so that they do not occupy too much space. We use the coloring to emphasize the success or failure of the test. Regarding the latter, we also utilize icdiff to clearly display all the differences between the expected and the actual output:

Failed Peachpie test on Travis CI


We have shown how to utilize Travis CI and .Net Core to build, run and test Peachpie on a Linux virtual server. As we can see, the combination of such technologies is not only fascinating, but also practically usable.

Posted on December 13, 2016, in category Information, Tutorial, tags: , , , , , , , , ,