[Svnmerge] Encoding problem in svnmerge commit message file

[Raman Gupta]

Romulo A. Ceccon wrote:
> Hi, Raman.
> 
> On Tue, Feb 19, 2008 at 8:06 PM, Raman Gupta <rocketraman at fastmail.fm> wrote:
>>  I spoke to Daniel Shahaf on #svn, and I got a slightly better
>>  understanding of svn behavior related to LC_ALL and commit log messages.
>>
>>  The attached patch is a slightly modified version of Romulo's that,
>>  allows svnmerge.py and svn to use the locale set by the user's
>>  environment, instead of trying to use the 'C' locale.
>>
>>  Could everyone (especially people in non-english locations) test it
>>  and verify it works in their environment?
> 
> On my Windows XP SP2 all test-cases pass with the "complete" patch,
> but testCommitMessageEncoding fails with the "simple" patch (as
> expected). I also tried them on a Ubuntu 6.06 LTS setup, and *both*
> pass all test-cases. Thanks, Raman.

Interesting... so on Windows XP SP2, svn or python or both are losing
the "default" encoding information somewhere. I think. You don't have
log-encoding set in your config file do you? See below.

> I should say that I'm not very confident about the
> testCommitMessageEncoding test-case, since it uses the same technique
> as svnmerge.py to find the console and text-file encodings. So if, for
> example, sys.stdout.encoding returned a completely wrong encoding, the
> test-case would still pass, although the user of svnmerge.py would get
> misencoded log messages. I don't know how the test-case could be
> enhanced, though, because svnmerge.py needs to make a guess about
> svn's behavior, but testCommitMessageEncoding cannot test without also
> making a guess.

Yeah, it would be great if you could use the svnmerge.py + the
"complete" patch in a real svn repo on your XP environment and verify
it works as expected. If you can also similarly test the "simple"
patch, and verify that the "simple" patch fails to do the right thing
in a real test case, that will also verify that the svnmerge_test.py
test case is correct and that the failure you are seeing on XP is valid.

There is one further situation here that neither of the patches
addresses. If the user has set log-encoding in their
.subversion/config, forcing svn to read the commit log file in a
non-default encoding, then the test will fail (and I believe
svnmerge.py will write the commit log with the wrong encoding).

For now, I just added this comment in the test case, however in
reality svnmerge.py should probably parse the config file and handle
the log-encoding setting:

# Check the message was properly encoded by svn (this will currently
# only work if the user config file does not override log-encoding

Cheers,
Raman Gupta