[enhancement] replace NTHREADS threading control with MAKEFLAGS, remove daal4py extension build parallelization

[icfaust]

Description

Follow on from #1196 which solves a leftover todo. This sets the cmake processes and cythonize multithreading based on the MAKEFLAGS -j flag. Written in a way that if a command line argument -j 2 is given, it will override MAKEFLAGS, but still maintains a default of the number of cpus if flag is not set in MAKEFLAGS.

Secondly, the parallelization of daal4py building was not possible with the previous code, and is not possible without significant rework. The custom build_ext setuptools Command object is replaced by the Cython version to extend capabilities in daal4py building.

Changes reduce building on github actions for DPC++ builds by 2-3 minutes, probably because the previous metric limited the number of processes.

No performance benchmarks necessary

---

PR should start as a draft, then move to ready for review state after CI is passed and all applicable checkboxes are closed.
This approach ensures that reviewers don't spend extra time asking for regular requirements.

You can remove a checkbox as not applicable only if it doesn't relate to this PR in any way.
For example, PR with docs update doesn't require checkboxes for performance while PR with any change in actual code should have checkboxes and justify how this code change is expected to affect performance (or justification should be self-evident).

Checklist to comply with before moving PR from draft:

PR completeness and readability

• I have reviewed my changes thoroughly before submitting this pull request.
• I have commented my code, particularly in hard-to-understand areas.
• I have updated the documentation to reflect the changes or created a separate PR with update and provided its number in the description, if necessary.
• Git commit message contains an appropriate signed-off-by string (see CONTRIBUTING.md for details).
• I have added a respective label(s) to PR if I have a permission for that.
• I have resolved any merge conflicts that might occur with the base branch.

Testing

• I have run it locally and tested the changes extensively.
• All CI jobs are green or I have provided justification why they aren't.
• I have extended testing suite if new functionality was introduced in this PR.

Performance

• I have measured performance for affected algorithms using scikit-learn_bench and provided at least summary table with measured data, if performance change is expected.
• I have provided justification why performance has changed or why changes are not expected.
• I have provided justification why quality metrics have changed or why changes are not expected.
• I have extended benchmarking suite and provided corresponding scikit-learn_bench PR if new measurable functionality was introduced in this PR.

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Flag	Coverage Δ
azure	76.78% <ø> (ø)
github	70.19% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

[icfaust]

/intelci: run

[icfaust]

/intelci: run

[icfaust]

/intelci: run

[icfaust]

/intelci: run

[david-cortes-intel]

@icfaust I'm seeing one issue now which doesn't happen in the master branch: a call to setup.py build will re-compile the cython part (daal4py) even if it was already built before through build_ext, whereas before it would reuse the build_ext result. No idea what causes this behavior.

[david-cortes-intel]

@icfaust I'm seeing one issue now which doesn't happen in the master branch: a call to setup.py build will re-compile the cython part (daal4py) even if it was already built before through build_ext, whereas before it would reuse the build_ext result. No idea what causes this behavior.

On a second look: perhaps it could be because Cython.Distutils.build_ext involves both .pyx -> .cpp and .cpp -> .so in addition to a manual call to 'cythonize' elsewhere, whereas the build_ext command it had before was only .cpp -> .so. Maybe it could just pass setuptools.command.build_ext if it works.

[david-cortes-intel]

LGTM, pending CI results.

[icfaust]

/intelci: run