Is there any way to designate a makefile as a "critical section" with ElectricAccelerator?
Is there a way to enforce that a certain Makefile can be run only on one agent at a time?
Basically due to other reasons, the same Makefile is getting executed from different parallel branches and causing conflicts, which in some cases have quite serious consequences for build performance. I am not able to change the way how we get there.
I think if I was able to say, when going through this Makefile allow only one agent to it at the time, would allow for this to be resolved peacefully, but not impact parallelism of the rest of the build.
something like:
#pragma oneagent
or
#pragma criticalsection
No, ElectricAccelerator has no means by which you can artificially limit the number of executions of a submake in this way. Truthfully the right way to address this problem is to use a good history file -- once you have run a "learning" build, emake will have the information to correctly serialize the many instances of this submake within your build, giving you exactly the performance behavior you want, so long as you manage your history file appropriately and make it available to future builds.
This question arised from the fact that learning build was taking 5 hours against about 20 min in good conditions.
The main drag is that if a conlict is detected the rerun of that job will be done in strictly serial order meaning that the compile time in my case just exploded.
I have since resolved the most of the conflict occurences (with help of EC) by putting make order rules on prerequisites which were causing conflicts, when run in parallel.
But it may be one area which EC could have a look, if the strict serialization of conflict rerun is really needed.
I know the suggesion is to maintain good history file, but that is overhead. It would be better if I did not need to. If the potential cost of conflict would be smaller i.e. the rerun could happen in parallel.
As developer I really do not want to manage any history. I just want to run emake :-) and enjoy quick builds what ever happens.
There could come a point, where it may be better to have consistently slower build (i.e switch to concurence) rather than not to know at which point the build just explodes, because the conflict happened at wrong place.
Just something to think about, how to make emake easier to use.
Thanks for the comments. We are constantly exploring ways to improve usability and performance. The very idea you describe is something we've been investigating for some time. Hopefully you'll see that show up in a future release!
One of the quick wins in this area could be use of jobcache, even for customers without it. If the reverted jobs were cached, it could be that when emake gets to job rerun, it would just use the cached result shrinking the time of serial run.
