Skip to content

StdErr is wrapped as ErrorRecord producing undesired effects #3813

Description

@vors

Steps to reproduce

Run some executable that outputs a long line to stderr and redirect it.
For example

python -c "from __future__ import print_function`nimport sys`nprint(range(100), file=sys.stderr)" *> 1

Real-life scenario that's highly impacted by that problem is searching in the logs.
When log file can be arbitrary split on the search tokens, you cannot reliable find anything in the logs.

Expected behavior

One line in the file 1

Actual behavior

Several lines in the output file.
Compare it with the redirection of stdout which works as expected.

PS /Users/svorobev/junk> python -c "from __future__ import print_function`nimport sys`nprint(range(100), file=sys.stdout)" *> 1                           
PS /Users/svorobev/junk> cat 1 | measure                                                                                                                  


Count    : 1
Average  : 
Sum      : 
Maximum  : 
Minimum  : 
Property : 



PS /Users/svorobev/junk> python -c "from __future__ import print_function`nimport sys`nprint(range(100), file=sys.stderr)" *> 1                           
PS /Users/svorobev/junk> cat 1 | measure                                                                                                                  


Count    : 3
Average  : 
Sum      : 
Maximum  : 
Minimum  : 
Property : 

Environment data

> $PSVersionTable
Name                           Value                                                                                                                     
----                           -----                                                                                                                     
PSVersion                      6.0.0-beta                                                                                                                
PSEdition                      Core                                                                                                                      
BuildVersion                   3.0.0.0                                                                                                                   
CLRVersion                                                                                                                                               
GitCommitId                    v6.0.0-beta.1                                                                                                             
OS                             Darwin 16.6.0 Darwin Kernel Version 16.6.0: Fri Apr 14 16:21:16 PDT 2017; root:xnu-3789.60.24~6/RELEASE_X86_64            
Platform                       Unix                                                                                                                      
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0...}                                                                                                   
PSRemotingProtocolVersion      2.3                                                                                                                       
SerializationVersion           1.1.0.1                                                                                                                   
WSManStackVersion              3.0 

Activity

  1. added this to the milestone on May 18, 2017
  2. vors commented on Jun 3, 2017

    @vors
    CollaboratorAuthor

    I looked into that, the gist of the problem is the following

    In GenerateOutOfBandData we are receiving View for error record, which is registered here.
    If we receive a View, it's automatically will be passed thru out format system, either as a complex object or as a list. That would force the screen width formatting on it.

    There are few ways we can go about it that come to my mind:

    • Put a special case hack 👎 - easy, but ugly
    • Consider to take away the formatting break logic from ComplexViewEntry maybe?
    • Subclass the NativeErrorRecord from ErrorRecord so we can have more granular control and put less ugly special case hack.

    Looking for opinions

  3. SteveL-MSFT commented on Jun 3, 2017

    @SteveL-MSFT
    Member

    Seems like option 3 is best. Whenever PowerShell works with native cmds we should keep the data raw for perf and compat reasons.

  4. iSazonov commented on Jun 4, 2017

    @iSazonov
    Collaborator

    Can the problem be in other streams?
    Can the problem will be resolved by native pipes #1908?

  5. iSazonov commented on Jun 5, 2017

    @iSazonov
    Collaborator

    I can't reproduce the bug with cmd.exe instead of python 😕

  6. vors commented on Jun 5, 2017

    @vors
    CollaboratorAuthor

    Ilya (@iSazonov) you may need to make your window smaller or output larger to observe it (essentially larger then 1 width of your window)

    Here is another repro with just powershell

    PS C:\dev\platyPS> powershell -Command '1..1000 | % { [System.Console]::Error.Write("1") }' *> 1
    PS C:\dev\platyPS> cat 1 | measure
    
    
    Count    : 12
    Average  :
    Sum      :
    Maximum  :
    Minimum  :
    Property :
    
    
    
  7. iSazonov commented on Jun 5, 2017

    @iSazonov
    Collaborator

    Thanks for the repo!

    I see that the formatting work well - we hit that line.
    If we remove the view the problem persists.
    So the problem is somewhere else.

  8. iSazonov commented on Jun 5, 2017

    @iSazonov
    Collaborator

    Maybe be the same problem #1491

  9. vors commented on Jun 5, 2017

    @vors
    CollaboratorAuthor

    I see that the formatting work well

    Let me elaborate the problem.

    If view != null here

    We are using ViewGenerator and it will apply logic for the window width.

    What we want is something like GenerateOutOfBandObjectAsToString(so)

  10. iSazonov commented on Jun 6, 2017

    @iSazonov
    Collaborator

    Thanks for clarify!

    Can we add new parameter in the view?

    CustomControl.Create(outOfBand: true, MaxWidth: Int.MaxValue)
  11. modified the milestones: 6.0.0-NiceToHave, on Oct 15, 2017
  12. modified the milestones: , on Oct 15, 2017
  13. changed the title [-]Redirecting stderr adds extra-line breaks[/-] [+]StdErr is wrapped as ErrorRecord producing undesired effects[/+] on Oct 19, 2017
  14. daxian-dbw commented on Nov 28, 2017

    @daxian-dbw
    Member

    Steve Lee (@SteveL-MSFT) Is this issue already fixed? The PR #5190 hasn't been merged, but I cannot repro with sergei (@vors)'s windows repro step on RC build:

    PS:1> pwsh -Command '1..1000 | % { [System.Console]::Error.Write("1") }' *> 1
    PS:2> cat 1 | measure
    
    
    Count    : 1
    Average  :
    Sum      :
    Maximum  :
    Minimum  :
    Property :
  15. SteveL-MSFT commented on Nov 29, 2017

    @SteveL-MSFT
    Member

    Dongbo Wang (@daxian-dbw) the original issue where line breaks were added at window width has been fixed. stderr is still being wrapped as error record, but it gets unwrapped I believe so it's just an optimization. I think we can close this since the original user facing issue is resolved.

  16. vors commented on Nov 30, 2017

    @vors
    CollaboratorAuthor

    Oh nice, didn't see that coming 💃

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Labels

Type

No type

Projects

No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions