[Svnmerge] [PATCH] Bidirectional merging patch for svnmerge.py

[Blair Zajac]

Giovanni Bajo wrote:
> Blair Zajac <blair at orcaware.com> wrote:
> 
>>>Here's some timing tests:
>>>
>>>Old method New method
>>>CPU time   16.99 sec   10.14 sec
>>>Clock time 1:09.80 sec   33.97 sec
>>>
>>>This logic reduced the number of revisions to check for reflected merges
>>>from 96 to 30.
>>>Much better.
> 
> 
> Good. many thanks for the numbers. How does that compare with no --bidi at
> all?

Taking the list of revisions that that --bidirectional merge ended up doing and 
passing that as a -r option to svnmerge.py without the --bidirectional merge 
resulted in the following numbers.  This just shows the overhead of the 
--bidirectional support.

Running with no --bidirectional, here's the complete numbers (all are averages 
over three runs, with running svnmerge.py once before these three runs to remove 
effects of reading files from disk and storing them in the buffer cache).
		
		No		Old		New
		Bidirectional	Bidirectional	Bidirectional
CPU time	 6.58 sec	  16.99 sec	10.14 sec
Clock time	18.10 sec	1:09.80 sec	33.97 sec

This isn't really a fair comparison in real life though, since I know I have 
bi-directional merges, and I'd rather spend twice the amount of clock time 
having svnmerge.py figure out which reflected revisions to ignore than me doing it.

The cost of enabling --bidirectional all the time should be done on a repository 
with no reflected revisions, and I'll guess that our 'svn log --verbose' 
optimization searching for changes on the merge source directory will remove all 
of the revisions to check.

Here are some numbers from a release branch project.

This is from an internal project, using svnmerge.py to merge all revisions from 
trunk to a 1.0 release branch, where there's already a 1.1 and a 1.2 release 
branch, there are a ton of revisions to merge.  The available revision range is 
89037:101644.  There are some 50 different projects in this repository though, 
so many are not for this one project.

Number of revisions merged from trunk to 1.0:		1269
Number of potential revision to check for reflection:	2

This merge took over 7 minutes of clock time and 173 sec of CPU time and 
checking 'svn diff' for two revisions is almost negligible difference in time.

I think we should turn the discussion around.  Enable --bidirectional by default 
unless we can show there are repositories with no reflected merges where it will 
slow it down significantly.

Do you have a branch you can try it with?

>>>+    # Running svn log with the --quiet and the --verbose options
>>>+    # prints all the paths to files and directory added, modified or
>>>+    # deleted in the commit without the log message.
>>>+    lines = launchsvn('log --quiet --verbose -r%s:%s "%s"' % (begin,
> 
> end,
> 
>>>url))
> 
> 
> I'd rather use --verbose only if bidi is active. I'm sure there is a penalty
> which is otherwise unnecessary.

I wouldn't assume as much.  We should see if the client has all the same 
information from the server, and only changes what to print.  If that's the 
case, then I'd keep the svnmerge.py code simpler.  If the server changes its 
response depending upon the svn log flags, then a case could be made to remove 
the --verbose flag.

> 
> 
>>>+                # Check to see if the revisions list changed for the
>>>specific
>>> +               # target branch involved -- if it has then it is
>>>a reflected
>>>+                # revision
> 
> 
> I thought about this. I believe the best solution is to extract the full
> property value (old and new) with a regular expression, and then pass it to
> a new function extracted from get_revlist_prop(). I believe we should have a
> single routine which knows how to analyze and parse the property.

Good idea.

Regards,
Blair