Skip to content

Perl crash during svn clone #274

Closed
Closed
Perl crash during svn clone#274
Labels
mcve-requiredup for grabs

Description

@uecasm
uecasm
opened on Aug 13, 2015

While trying to git svn clone a large private repository using Git-2.4.5.1-4th-release-candidate-64-bit, it gets some way through the initial fetch and then crashes with:

  0 [main] perl 12640 cygwin_exception::open_stackdumpfile: Dumping stack trace to perl.exe.stackdump

The dumpfile contains the following:

Exception: STATUS_ACCESS_VIOLATION at rip=0048360C537
rax=00000006020E4908 rbx=000000005219E248 rcx=000000060003A540
rdx=0000000000000000 rsi=000000000000F4FD rdi=0000000000000004
r8 =0000000000000000 r9 =0000000000000000 r10=0000000000240000
r11=000000048D785FBA r12=0000000000000003 r13=000006FFFFE8A800
r14=0000000601FFAB20 r15=000006FFFFE8A818
rbp=000000000007A7F0 rsp=000000000023BDE0
program=C:\Program Files\Git\usr\bin\perl.exe, pid 12640, thread main
cs=0033 ds=002B es=002B fs=0053 gs=002B ss=002B

I realise this is probably not directly helpful because I can't share the repo so you can't directly reproduce it, but I'm not sure how to diagnose this further. Note that a colleague on a different PC using Git-2.4.6-5th-release-candidate-64-bit is getting similar crashes.

Activity

  1. nalla commented on Aug 13, 2015

    @nalla
    nalla
    on Aug 13, 2015

    This is not windows specific. Even under linux if the repository is large enough, git-svn will fail. try calling successive git svn fetch calls to continue.

  2. uecasm commented on Aug 13, 2015

    @uecasm
    uecasm
    on Aug 13, 2015
    Author

    A few other things that are possibly of interest:

    1. I'm using the -r nnnnn:HEAD option (where nnnnn is from a day or two ago) to clone in order to produce an abbreviated history -- as a result the first commit it imports is quite large, and it's crashing during this first revision import, so successive calls don't help.

    2. Running the same clone operation using Git-2.4.6-5th-release-candidate-32-bit resulted in the following crash dump (also I still had to rebase several SVN DLLs before it would run at all):

      Exception: STATUS_ACCESS_VIOLATION at eip=64B0CC7F
eax=81CDF488 ebx=521BA170 ecx=FFFDF000 edx=00000000 esi=00066E44 edi=FFD76E50
ebp=FFD76E50 esp=0028C380 program=C:\Program Files (x86)\git\usr\bin\perl.exe, pid 17404, thread main
cs=0023 ds=002B es=002B fs=0053 gs=002B ss=002B
Stack trace:
Frame     Function  Args
End of stack trace

    3. At some previous time (about 9 months and 10,000 SVN revisions ago) a similar clone on the same repository succeeded, while using GfW 1.9.5.

  3. nalla commented on Aug 13, 2015

    @nalla
    nalla
    on Aug 13, 2015

    I'm using the -r nnnnn:HEAD option (where nnnnn is from a day or two ago) to clone in order to produce an abbreviated history -- as a result the first commit it imports is quite large.

    That large commit is most probably the cause. The change set is so big that the data perl.exe is trying to analyze does not fit into the memory it can allocate.

  4. uecasm commented on Aug 13, 2015

    @uecasm
    uecasm
    on Aug 13, 2015
    Author

    Shouldn't it work on the 64-bit version then? Memory limits shouldn't be an issue there.

    Hmm. So I tried doing a clone from the same earlier rev# as 9 months ago, and it seemed to run for a lot longer before finally crashing (still during the first commit though). There were 4 perl processes and only one of them was using memory, but even that peaked at only 90 MB or so, so it doesn't seem likely that memory exhaustion would be the culprit.

    It bothers me that a Perl script can cause Perl itself to crash without a useful backtrace, though.

    Exception: STATUS_ACCESS_VIOLATION at eip=64B0CC7F
eax=848D3F20 ebx=521BA170 ecx=FFFDF000 edx=00000000 esi=00172FD4 edi=FFA72FE0
ebp=FFA72FE0 esp=0028C380 program=C:\Program Files (x86)\git\usr\bin\perl.exe, pid 16980, thread main
cs=0023 ds=002B es=002B fs=0053 gs=002B ss=002B
Stack trace:
Frame     Function  Args
End of stack trace

    Since the EIP values seem to be the same in all crashes (on the same version, of course), I took a look at the base addresses; apparently that EIP is in _Delta.dll, in case that helps.

  5. dscho
    added
    mcve-required
    on Aug 13, 2015

  6. dscho commented on Aug 13, 2015

    @dscho
    dscho
    on Aug 13, 2015
    Member

    Sorry, without any way to reproduce here, there is nothing I can do.

  7. uecasm commented on Aug 13, 2015

    @uecasm
    uecasm
    on Aug 13, 2015
    Author

    According to a debugger, the crashing IP is at:

    C:\Program Files (x86)\git\usr\lib\perl5\vendor_perl\auto\SVN\_Delta\_Delta.dll
_Delta!wrap_svn_txdelta_apply+0x1cf

    And now I've completely run out of ideas of ways to help track this down.

  8. dscho commented on Aug 13, 2015

    @dscho
    dscho
    on Aug 13, 2015
    Member

    I really think that the only way forward here is to give somebody with strong debugging fu (and enough time, so do not look at me) access to said repository.

  9. dscho commented on Aug 13, 2015

    @dscho
    dscho
    on Aug 13, 2015
    Member

    C:\Program Files (x86)\

    And didn't you intend to use the 64-bit version of Git for Windows?

  10. nalla commented on Aug 13, 2015

    @nalla
    nalla
    on Aug 13, 2015

    Do you have files like Git_git_blob_xxxx_x_xxxxxx or Git_svn_delta_xxxx_x_xxxxxx in your .git folder?

  11. uecasm commented on Aug 13, 2015

    @uecasm
    uecasm
    on Aug 13, 2015
    Author

    And didn't you intend to use the 64-bit version of Git for Windows?

    Yes, but as I mentioned earlier I tried switching to the 32-bit version to see if it helped with this. It didn't, but I haven't switched back yet in order to keep the crash traces consistent.

    In theory that symbol trace above might be enough to give a hint to whoever built the SVN DLLs that you're using, if you can tell them which build was used in that particular GfW release. I realise there's probably not much else you can do without a way to reproduce it. (I tried finding a big public repository and seeing if a similar command crashes it, but no such luck yet. Big repos seem hard to find.)

    Do you have files like Git_git_blob_xxxx_x_xxxxxx or Git_svn_delte_xxxx_x_xxxxxx in your .git folder?

    Yes, they (and a Git_svn_hash file) exist while it's downloading and seem to occasionally have contents, but when it crashed they still exist but seem to be empty.

  12. uecasm commented on Aug 13, 2015

    @uecasm
    uecasm
    on Aug 13, 2015
    Author

    Well, hey. It took quite a long time to get there, but running this command did indeed cause a crash. Perhaps this will be reproducable?

    git svn clone -r 1400000:HEAD --prefix=origin/ svn://anonsvn.kde.org/home/kde/trunk kde

    Still on Git-2.4.6-5th-release-candidate-32-bit (not that it seems to make much difference).

    Exception: STATUS_ACCESS_VIOLATION at eip=64B0CC7F
eax=818923A8 ebx=521BA170 ecx=FFFDE000 edx=00000000 esi=0004A2DC edi=FFE4A2E8
ebp=FFE4A2E8 esp=0028C3D0 program=C:\Program Files (x86)\git\usr\bin\perl.exe, pid 8684, thread main
cs=0023 ds=002B es=002B fs=0053 gs=002B ss=002B
Stack trace:
Frame     Function  Args
End of stack trace

    cding into the directory and running git svn fetch -r 1400000:HEAD does start it running again, but I'm not sure whether it's resuming where it left off or whether it's starting again.

  13. dscho commented on Aug 14, 2015

    @dscho
    dscho
    on Aug 14, 2015
    Member

    I'm not sure whether it's resuming where it left off or whether it's starting again.

    As pointed out by @nalla, it is safe to fetch after an access violation.

  14. dscho commented on Aug 27, 2015

    @dscho
    dscho
    on Aug 27, 2015
    Member

    Still no way to reproduce, eh?

  15. maoueh commented on Aug 27, 2015

    @maoueh
    maoueh
    on Aug 27, 2015

    @dscho Facing a similar issue today trying to git svn fetch url svn://sourceware.org/svn/kawa.The initial clone was performed using 1.9.5 version and now, trying to sync it with version 2.5.0 64 bits.

    Got 0 [main] perl 12640 cygwin_exception::open_stackdumpfile: Dumping stack trace to perl.exe.stackdump at multiple times during git svn fetch and they always recovered at second attempt.

    Now, I'm stuck at revision r8526 and git svn fetch simply always crash on revision r8527. Strange thing is sometimes for this commit I get crash dump message, and other time I get this one:

    Connection timed out: Error retrieving REPORT: Connection timed out at /mingw64/share/perl5/site_perl/Git/SVN/Ra.pm line 308.

    Tried to reproduce with:

    git svn clone -r 8526:8528 --prefix=origin/ svn://sourceware.org/svn/kawa/trunk kawa

    But this is working as expected. I will launch a full clone using newer 2.5.0 version to see if it works. Cannot start it right now, probably tonight on another computer.

    I could upload my full working copy to see if you can reproduce using it if you want. I want to investigate further the crash dump but don't have time right now. Hoping to find a slot next week. Any hints one debugging this?

    Extra info just in case:

    • Windows 7 x86_64
    • Git for Windows 2.5.0.windows1 (installed inside a MSYS2 proper)
    • SVN 1.8.13
    • Perl 5.22.0

  16. 69 remaining items

  17. rimrul commented on Feb 20, 2019

    @rimrul
    rimrul
    on Feb 20, 2019
    Member

    @josesimoes That sounds more like #1993. You're probably cloning via HTTPS and according to the documentation git svn clone runs git svn init and git svn fetch. That hanging issue should be fixed in the next Git for Windows release.

  18. dscho commented on Feb 27, 2019

    @dscho
    dscho
    on Feb 27, 2019
    Member

    Can y'all please test with v2.21.0? It should be fixed via #1993.

  19. breakersun commented on Mar 15, 2019

    @breakersun
    breakersun
    on Mar 15, 2019

    still got problem with v2.21.0.

    Initialized empty Git repository in G:/xxx/.git/
      1 [main] perl 2424 cygwin_exception::open_stackdumpfile: Dumping stack trace to perl.exe.stackdump```


perl.exe.stackdump:

Exception: STATUS_ACCESS_VIOLATION at rip=00000000000
rax=0000000000000000 rbx=0000000601130D58 rcx=0000000601139D98
rdx=00000006011420D8 rsi=0000000000000000 rdi=0000000000000011
r8 =0000000000000000 r9 =0000000601177938 r10=0000000100000000
r11=0000000601177938 r12=00000000FFFFC218 r13=0000000601130DE8
r14=0000000601139D98 r15=0000000000000000
rbp=00000000FFFFC210 rsp=00000000FFFFC1A8
program=D:\Program Files\Git\usr\bin\perl.exe, pid 2424, thread main
cs=0033 ds=002B es=002B fs=0053 gs=002B ss=002B
Stack trace:
Frame        Function    Args
End of stack trace

  20. dscho commented on Mar 15, 2019

    @dscho
    dscho
    on Mar 15, 2019
    Member

    still got problem with v2.21.0.

    Bummer.

    @breakersun please understand that these issues are extremely involved to investigate. So much so that I, for example, cannot afford to invest the time.

    Hopefully you can, or maybe you know somebody competent enough who can, an are able to bribe them into doing it.

  21. breakersun commented on Mar 18, 2019

    @breakersun
    breakersun
    on Mar 18, 2019

    still got problem with v2.21.0.

    Initialized empty Git repository in G:/xxx/.git/
      1 [main] perl 2424 cygwin_exception::open_stackdumpfile: Dumping stack trace to perl.exe.stackdump```


perl.exe.stackdump:

Exception: STATUS_ACCESS_VIOLATION at rip=00000000000
rax=0000000000000000 rbx=0000000601130D58 rcx=0000000601139D98
rdx=00000006011420D8 rsi=0000000000000000 rdi=0000000000000011
r8 =0000000000000000 r9 =0000000601177938 r10=0000000100000000
r11=0000000601177938 r12=00000000FFFFC218 r13=0000000601130DE8
r14=0000000601139D98 r15=0000000000000000
rbp=00000000FFFFC210 rsp=00000000FFFFC1A8
program=D:\Program Files\Git\usr\bin\perl.exe, pid 2424, thread main
cs=0033 ds=002B es=002B fs=0053 gs=002B ss=002B
Stack trace:
Frame        Function    Args
End of stack trace

    I figured it out and find a work around for my issue.
    The reason is that SVN repo must be accessed via a http/https proxy.
    for git-svn for windows, you must add proxy information in $home/.subversion/servers.(c:\Users\ASUS\.subversion\servers), find global sector and uncommet http-proxy-host and http-proxy-host.
    on windows, git-svn proxy settings is not same as git proxy settings, that's my problem.

  22. dscho commented on Mar 21, 2019

    @dscho
    dscho
    on Mar 21, 2019
    Member

    @breakersun whoa. Good analysis! But the absence of a proxy should not let git svn crash...

    Is there any chance that you can debug this, say, by rebuilding the subversion package, patching in debug print statements?

  23. breakersun commented on Mar 27, 2019

    @breakersun
    breakersun
    on Mar 27, 2019

    @dscho Thank you. I'd like to help. But I need to do some study first cause this is not my expertise.

  24. frankggyy commented on May 6, 2019

    @frankggyy
    frankggyy
    on May 6, 2019

    still got problem with v2.21.0.
    Initialized empty Git repository in G:/xxx/.git/
    1 [main] perl 2424 cygwin_exception::open_stackdumpfile: Dumping stack trace to perl.exe.stackdump```

    perl.exe.stackdump:

    Exception: STATUS_ACCESS_VIOLATION at rip=00000000000
    rax=0000000000000000 rbx=0000000601130D58 rcx=0000000601139D98
    rdx=00000006011420D8 rsi=0000000000000000 rdi=0000000000000011
    r8 =0000000000000000 r9 =0000000601177938 r10=0000000100000000
    r11=0000000601177938 r12=00000000FFFFC218 r13=0000000601130DE8
    r14=0000000601139D98 r15=0000000000000000
    rbp=00000000FFFFC210 rsp=00000000FFFFC1A8
    program=D:\Program Files\Git\usr\bin\perl.exe, pid 2424, thread main
    cs=0033 ds=002B es=002B fs=0053 gs=002B ss=002B
    Stack trace:
    Frame Function Args
    End of stack trace

    I figured it out and find a work around for my issue.
    The reason is that SVN repo must be accessed via a http/https proxy.
    for git-svn for windows, you must add proxy information in $home/.subversion/servers.(c:\Users\ASUS.subversion\servers), find global sector and uncommet http-proxy-host and http-proxy-host.
    on windows, git-svn proxy settings is not same as git proxy settings, that's my problem.

    Fixed in my environment.

  25. vitaliyg2 commented on Jun 24, 2019

    @vitaliyg2
    vitaliyg2
    on Jun 24, 2019

    Same error when try to clone SVN repo over TCP tunnel.
    I can access repo using browser and was able to clone it using the SVN client (so this is not a network error), but can't clone because of this error.
    This is relatively small repo.
    Subsequent git svn fetch results in the same error.

    git version 2.22.0.windows.1

    0 [main] perl 471 cygwin_exception::open_stackdumpfile: Dumping stack trace to perl.exe.stackdump

    Exception: STATUS_ACCESS_VIOLATION at rip=00000000000
    rax=0000000000000000 rbx=0000000601136838 rcx=000000060113F878
    rdx=0000000601147BB8 rsi=0000000000000000 rdi=0000000000000011
    r8 =0000000000000000 r9 =000000060117D368 r10=0000000100000000
    r11=000000060117D368 r12=00000000FFFFC248 r13=00000006011368C8
    r14=000000060113F878 r15=0000000000000000
    rbp=00000000FFFFC240 rsp=00000000FFFFC1D8
    program=D:\Programs\Git\usr\bin\perl.exe, pid 1680, thread main
    cs=0033 ds=002B es=002B fs=0053 gs=002B ss=002B
    Stack trace:
    Frame Function Args
    End of stack trace

  26. fuzhibo commented on May 22, 2020

    @fuzhibo
    fuzhibo
    on May 22, 2020

    Same error when try to clone SVN repo over TCP tunnel, then I use following config:

    [core]
       repositoryformatversion = 0
       filemode = false
       bare = false
       logallrefupdates = true
       ignorecase = true
       longpaths = true
       packedGitLimit = 256m
       packedGitWindowsSize = 256m
[http]
       postBuffer = 534288000
[pack]
       deltaCacheSize = 256m
       packSizeLimit = 256m
       windowMemory = 1024m

    It worked, but failure with command git svn rebase. The git version is git version 2.26.0.windows.1. Error report is:

    Exception: STATUS_ACCESS_VIOLATION at rip=00000000000
rax=0000000000000000 rbx=0000000000000011 rcx=00000006011337F8
rdx=000000060113B838 rsi=00000000FFFFC260 rdi=00000000FFFFC268
r8 =0000000000000000 r9 =000000060113DBB8 r10=00000006011337F8
r11=000000060113DBB8 r12=000000060112A7B8 r13=00000000FFFFC278
r14=00000000FFFFC270 r15=0000000000000000
rbp=000000060112A848 rsp=00000000FFFFC208
program=C:\Program Files\Git\usr\bin\perl.exe, pid 50, thread main
cs=0033 ds=002B es=002B fs=0053 gs=002B ss=002B
Stack trace:
Frame        Function    Args
End of stack trace

  27. dscho commented on May 22, 2020

    @dscho
    dscho
    on May 22, 2020
    Member

    @fuzhibo does it work for you if you use Git in WSL?

  28. fuzhibo commented on Jun 30, 2020

    @fuzhibo
    fuzhibo
    on Jun 30, 2020

    @fuzhibo does it work for you if you use Git in WSL?

    It seems worked on WSL. However I use command git svn clone --username=xxx -r xxx:HEAD --trunk=xxx <svn url>. @dscho

  29. carlbjor commented on Jul 9, 2020

    @carlbjor
    carlbjor
    on Jul 9, 2020

    Adding svn proxy settings helped in my case, as @breakersun suggested in a previous comment

    https-proxy-host=
http-proxy-port=

  30. dscho commented on Oct 15, 2021

    @dscho
    dscho
    on Oct 15, 2021
    Member

    Working on bugs in git svn is extremely involved. Combined with the fact that fewer and fewer people seem to use git svn, I fear that there is little sense in my working on this (or for that matter, any other developer competent enough to debug MSYS2 runtime issues).

    I will therefore simply close this ticket.

  31. dscho
    closed this as completedon Oct 15, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    mcve-requiredup for grabs

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions