Repository navigation
Inconsistent loading order of packages and their parts #7056
Description
Describe the bug
It seems there is different loading order of plugins and their parts if using empty vimrc / packload / packadd / packadd!
To Reproduce
I've created https://github.com/sheerun/vim-packages-bug that should better illustrate the problem. You can run there bash script.sh to produce following output:
- Putting package in
~/.vim/pack/*/start/*without pack commands in ~/.vimrc
loading plugin/*.vim after native vim plugins
loading ftdetect/*.vim after native filetype.vim
loading after/plugin/*.vim after native vim plugins
Press ENTER or type command to continue
- Putting package in
~/.vim/pack/*/start/*withpackload
loading plugin/*.vim before native vim plugins
loading ftdetect/*.vim after native filetype.vim
loading plugin/*.vim before native vim plugins
loading after/plugin/*.vim after native vim plugins
Press ENTER or type command to continue
- Putting package in
~/.vim/pack/*/start/*withpackadd vim-packages-bugin vimrc
loading plugin/*.vim before native vim plugins
loading ftdetect/*.vim after native filetype.vim
loading plugin/*.vim before native vim plugins
loading plugin/*.vim after native vim plugins
loading ftdetect/*.vim after native filetype.vim
loading after/plugin/*.vim after native vim plugins
Press ENTER or type command to continue
- Putting package in
~/.vim/pack/*/start/*withpackadd! vim-packages-bugin vimrc
loading plugin/*.vim before native vim plugins
loading plugin/*.vim after native vim plugins
loading ftdetect/*.vim after native filetype.vim
loading after/plugin/*.vim after native vim plugins
Press ENTER or type command to continue
- Putting package in
~/.vim/pack/*/opt/*withpackadd vim-packages-bugin vimrc
loading plugin/*.vim before native vim plugins
loading ftdetect/*.vim after native filetype.vim
loading plugin/*.vim before native vim plugins
loading after/plugin/*.vim after native vim plugins
Press ENTER or type command to continue
- Putting package in
~/.vim/pack/*/opt/*withpackadd! vim-packages-bugin vimrc
loading plugin/*.vim before native vim plugins
loading after/plugin/*.vim after native vim plugins
Press ENTER or type command to continue
To summarize
pluginscan be loaded before or after vim plugins, possibly multiple timesftdetectis loaded always after vim filetypes, possibly multiple times (all other package managers always load ftdetect before vim native files..)- In some cases no part of plugin is loaded before vim parts
Things get even worse if multiple packages are used, because then they can be loaded in alphabetical order (no pack commands), in defined order (packadd commands), or reverse defined order (packadd! commands), and all combinations of these. Order of loading is also affected if some plugins are calling filetype plugin indent on within themselves (like vim-sensible). Also all other package managers are loading filetype.vim of plugin while vim8 packages do not, ever.
Honestly it's pretty overwhelming for someone who would like to depend on some order of loading in package manager (the vim-polyglot plugin I'm developing needs to be loaded before vim's filetype.vim, and other plugins overriding its autocommands).
Environment (please complete the following information):
VIM - Vi IMproved 8.2 (2019 Dec 12, compiled Sep 16 2020 23:47:44)
macOS version
Included patches: 1-1700
Compiled by Homebrew
Huge version without GUI. Features included (+) or not (-):
+acl -farsi +mouse_sgr +tag_binary
+arabic +file_in_path -mouse_sysmouse -tag_old_static
+autocmd +find_in_path +mouse_urxvt -tag_any_white
+autochdir +float +mouse_xterm -tcl
-autoservername +folding +multi_byte +termguicolors
-balloon_eval -footer +multi_lang +terminal
+balloon_eval_term +fork() -mzscheme +terminfo
-browse +gettext +netbeans_intg +termresponse
++builtin_terms -hangul_input +num64 +textobjects
+byte_offset +iconv +packages +textprop
+channel +insert_expand +path_extra +timers
+cindent +ipv6 +perl +title
-clientserver +job +persistent_undo -toolbar
+clipboard +jumplist +popupwin +user_commands
+cmdline_compl +keymap +postscript +vartabs
+cmdline_hist +lambda +printer +vertsplit
+cmdline_info +langmap +profile +virtualedit
+comments +libcall -python +visual
+conceal +linebreak +python3 +visualextra
+cryptv +lispindent +quickfix +viminfo
+cscope +listcmds +reltime +vreplace
+cursorbind +localmap +rightleft +wildignore
+cursorshape +lua +ruby +wildmenu
+dialog_con +menu +scrollbind +windows
+diff +mksession +signs +writebackup
+digraphs +modify_fname +smartindent -X11
-dnd +mouse -sound -xfontset
-ebcdic -mouseshape +spell -xim
+emacs_tags +mouse_dec +startuptime -xpm
+eval -mouse_gpm +statusline -xsmp
+ex_extra -mouse_jsbterm -sun_workshop -xterm_clipboard
+extra_search +mouse_netterm +syntax -xterm_save
system vimrc file: "$VIM/vimrc"
user vimrc file: "$HOME/.vimrc"
2nd user vimrc file: "~/.vim/vimrc"
user exrc file: "$HOME/.exrc"
defaults file: "$VIMRUNTIME/defaults.vim"
fall-back for $VIM: "/usr/local/share/vim"
Compilation: clang -c -I. -Iproto -DHAVE_CONFIG_H -DMACOS_X -DMACOS_X_DARWIN -g -O2 -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=1
Linking: clang -L. -fstack-protector-strong -L/usr/local/lib -L/usr/local/opt/libyaml/lib -L/usr/local/opt/[email protected]/lib -L/usr/local/opt/readline/lib -L/usr/local/lib -o vim -lncurses -liconv -lintl -framework AppKit -L/usr/local/opt/lua/lib -llua5.3 -mmacosx-version-min=10.15 -fstack-protector-strong -L/usr/local/lib -L/usr/local/Cellar/perl/5.32.0/lib/perl5/5.32.0/darwin-thread-multi-2level/CORE -lperl -lm -lutil -lc -L/usr/local/opt/[email protected]/Frameworks/Python.framework/Versions/3.8/lib/python3.8/config-3.8-darwin -lpython3.8 -framework CoreFoundation -lruby.2.7
Activity
sheerun commentedon Sep 30, 2020
Also ftdetect can be not loaded if packadd! is used and plugin is in opt (but it works if it's in start)....
sheerun commentedon Sep 30, 2020
Oh, plus vim and nvim have different behavior for each of these: k-takata/minpac#118 (comment) (minpac depends on vim8 / neovim packages to load plugins instead of modiifying rtp like vim-plug or vundle)
I mean in current form vim8 packages are pretty much unstable, and actually much harder to work with than any other solution available. I think they need v2.0 with much simpler logic, something like this:
~/.vim/pack/<packagename>/<pluginname>/{indent,plugin,ftdetect,after,...}pack vim-somethingafterdirectories of plugins are added at the end of rtp, also in chronological order (i.e. defined by order ofpackcommands in ~/.vimrc).packloadwhich can load plugins in unstable order (alphabetical, reverse alphabetical)packcommand in theirplugin/*scripts)plugis called (with after directories deferred). This allows for easy overridding of settings set by plugins directly in ~/.vimrcbrammool commentedon Oct 1, 2020
"the vim-polyglot plugin I'm developing needs to be loaded before vim's filetype.vim". That is a very difficult restriction you are making. The user may have "syntax on" somewhere you don't expect it. Why not remove this dependency?
Also, packages should not depend on the loading order. :packload first adds all the directories to 'runtimepath', so that when there are dependencies it should be possible to find them.
sheerun commentedon Oct 1, 2020
Well, for me it's difference between 60ms and 10ms of loading time, because that's how much hundreds of
au!calls take (aucalls are pretty fast), and I need to run them to override vim defaults if I load after vim. I can (and will) program polyglot to useau!instead ofauif for some reason vim files loaded first (which will satisfy "packages should not depend on the loading order"), but it'll always have performance penalty. With current implementation of vim8 packages it's pretty much guaranteed that vim-polyglot loads later, while for all other plugin managers it's pretty much guaranteed that it'll always load first.brammool commentedon Oct 1, 2020
For filetypes it's basically "last one wins", so why do you need to remove what was added before?
Specifically, is this about the order in which autocommands are added, or the order of 'runtimepath', which changes in which order the plugins are found?
sheerun commentedon Oct 1, 2020
runtimepath works well for all files except initial ftdetect files, or rather filetype.vim (e.g. ruby.vim of vim-polyglot is properly loaded always before vim's). Indeed there's "last wins", but changing filetype multiple times for loaded file has its own performance penalty, so it's best handled by ensuring correct filetype is applied in the first place (vim uses
setfwchich is "first one wins").sheerun commentedon Oct 1, 2020
I mean I can hack anything as long as something is loaded before vim's runtime files. Current implementation doesn't load anything at all, so there's no way around it. I basically must live with vim runtime files being sourced first, and I cannot do anything about it, this is the gist of this issue.
brammool commentedon Oct 1, 2020
sheerun commentedon Oct 1, 2020
Yes indeed, I've mentioned I can use
au!but it has heavy performance penalty. 60ms vs 10ms loading timesheerun commentedon Oct 1, 2020
I've decided to not use
au!for each overridden filetype, butau! filetypedetect+ basically copy all vim's filetype.vim into vim-polyglot, which finally I know will load last :PAs of now the issue is twofold:
au!can't be used by other plugins because it depends on load order of plugins, and also it requires that arguments are exactly the same which they often aren't. for example one plugin can useau! BufRead Makefile.am setf make, when second usesau! BufRead [Mm]akeifile.am setf makefile(second au! won't override first one, and filetype will be set to make)This could all be solved by instructing vim to source second plugin first. which currently isn't encouraged ("packages should not depend on the loading order") or even possible to enforce with vim8 packages
brammool commentedon Oct 3, 2020
brammool commentedon Oct 3, 2020
sheerun commentedon Oct 3, 2020
In case vim's filetype.vim is sourced first, I'm running
au! filetypedetectwhich quickly clears any autocommands set by filetype.vim, and then I "source" filetype.vim again (in reality I've just copied some code from filetype.vim)In case vim's filetype.vim is sourced last, thanks to
let did_load_filetypes = 1set by vim-polyglot, it makes sourcing vim's filetype.vim a no-op 0ms, instead of 25ms (optimization which I cannot do iffiletype.vimof vim is loaded first).It isn't one-time delay, it's difference of calling hundreds of
au, vs calling analogical hundreds ofau!commands (I'm no longer doing that, what I'm doing now is just singleau! filetypedetect, which is fast and doesn't need to be improved).Currently without vim-polyglot loading time is as follows:
With vim-polyglot loaded after filetype.vim loading time is as follows:
If vim-polyglot is loaded before filetype.vim, then loading time is as follows:
So using vim-polyglot makes startup time faster than native, if sourced in correct order, and slower than native if in incorrect order.