Skip to content
This repository has been archived by the owner on Sep 20, 2021. It is now read-only.

Example benchmark #96

Open
wants to merge 2 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,2 +1,3 @@
/vendor/
/composer.lock
/_storage
88 changes: 88 additions & 0 deletions Test/Benchmark/RulerBench.php
Original file line number Diff line number Diff line change
@@ -0,0 +1,88 @@
<?php

/**
* Hoa
*
*
* @license
*
* New BSD License
*
* Copyright © 2007-2016, Hoa community. All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions are met:
* * Redistributions of source code must retain the above copyright
* notice, this list of conditions and the following disclaimer.
* * Redistributions in binary form must reproduce the above copyright
* notice, this list of conditions and the following disclaimer in the
* documentation and/or other materials provided with the distribution.
* * Neither the name of the Hoa nor the names of its contributors may be
* used to endorse or promote products derived from this software without
* specific prior written permission.
*
* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
* AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
* ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDERS AND CONTRIBUTORS BE
* LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
* CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
* SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
* INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
* CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
* ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
* POSSIBILITY OF SUCH DAMAGE.
*/

namespace Hoa\Ruler\Test\Benchmark;

/**
* @Revs(10)
* @Iterations(10)
* @OutputTimeUnit("milliseconds")
*/
class RulerBench
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we can rename RulerBench to Ruler since this is an Benchmark namespace.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Historically this was a mandatory suffix (like Test in PHPUnit), but not since 0.12 so, can do.

{
private $ruler;
private $context;

public function setUpAssert()
{
$this->ruler = new \Hoa\Ruler\Ruler();

// 2. Create a context.
$this->context = new \Hoa\Ruler\Context();
$this->context['group'] = 'customer';
$this->context['points'] = function () {
return 42;
};
}

public function provideRules()
{
return [
[
'rule' => 'group in ["customer", "guest"] and points > 30'
],
[
'rule' => 'group in ["customer", "guest"]'
],
[
'rule' => 'points > 30'
],
];
}

/**
* Benchmark parsing with rules of different complexity.
* Executes the method once (warmup) before measuring time.
*
* @Warmup(1)
* @BeforeMethods({"setUpAssert"})
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you have a default setUp method?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can have an abstract class which defines a setUp method (and its annotation)

* @ParamProviders({"provideRules"})
*/
public function case_assert($params)
{
$this->ruler->assert($params['rule'], $this->context);
}
}
3 changes: 2 additions & 1 deletion composer.json
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,8 @@
"hoa/visitor" : "~2.0"
},
"require-dev": {
"hoa/test": "~2.0"
"hoa/test": "~2.0",
"phpbench/phpbench": "^0.12.2"
},
"autoload": {
"psr-4": {
Expand Down
12 changes: 12 additions & 0 deletions phpbench.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
{
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we move this file in a single place, like in Hoa\Test, it will avoid to have it inside each library?
Note that we will probably create the hoa test:bench or update the hoa test:run command, and thus I suppose we could link this phpbench.json file with an argument. We already do this with hoa test:run where we already compute the atoum command (https://github.com/hoaproject/Test/blob/f80f083a0d1cc7a044fdbca10ea46cdcdbef4c38/Bin/Run.php#L291-L326) with all the bootstrap file, configuration file etc.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The issue would be if you wanted to define f.e. a custom report.

However, most, if not all, of the configuration options can be specified at the CLI, so if you wrap it in hoa test:bench then we can ommit the phpbench.json file and include it only when the library has specific requirements.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We could also add a Hoa phpbench extension which programatically adds reports if required.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, sounds interesting. Can you tell me more about it please?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The Extension instances for PHPBench are like modules for the DI container. It should be possible to add new configurations and overwrite existing ones actually this is disallowed currently.

The extensions can be enabled via. the command line or in phpbench.json as with the example

Copy link
Member

@Hywan Hywan Sep 26, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would you like to give it a try? Please, feel free to say no!

"bootstrap": "vendor/autoload.php",
"path": "Test/Benchmark",
"subject_pattern": "^case_.*",
"reports": {
"hoa": {
"extends": "aggregate",
"break": ["benchmark", "subject"],
"cols": [ "params", "revs", "its", "mem_peak", "best", "mean", "mode", "rstdev" ]
}
}
}