I am writing a PowerShell script in version 5.1 on Windows 10 that gets certain pieces of information about a local system ( and eventually its subnets ) and outputs them into a text file. At first, I had all of the aspects in a single function. I ran into output issues when outputting getUsersAndGroups and getRunningProcesses functions, where output from getUsersAndGroups would be injected into the output of getRunningProcesses.
The two functions are:
# Powershell script to get various properties and output to a text file
Function getRunningProcesses()
{
# Running processes
Write-Host "Running Processes:
------------ START PROCESS LIST ------------
"
Get-Process | Select-Object name,fileversion,productversion,company
Write-Host "
------------- END PROCESS LIST -------------
"
}
Function getUsersAndGroups()
{
# Get Users and Groups
Write-Host "Users and Groups:"
$adsi = [ADSI]"WinNT://$env:COMPUTERNAME"
$adsi.Children | where {$_.SchemaClassName -eq 'user'} | Foreach-Object {
$groups = $_.Groups() | Foreach-Object {$_.GetType().InvokeMember("Name", 'GetProperty', $null, $_, $null)}
$_ | Select-Object #{n='Username';e={$_.Name}},#{n='Group';e={$groups -join ';'}}
}
}
getRunningProcesses
getUsersAndGroups
When I call getUsersAndGroups after getRunningProcesses, the output looks like this ( does not output getUsersAndGroups at all ):
Running Processes:
------------ START PROCESS LIST ------------
Name FileVersion ProductVersion Company
---- ----------- -------------- -------
armsvc
aswidsagenta
audiodg
AVGSvc
avgsvca
avguix 1.182.2.64574 1.182.2.64574 AVG Technologies CZ, s.r.o.
conhost 10.0.14393.0 (rs1_release.160715-1616) 10.0.14393.0 Microsoft Corporation
csrss
csrss
dasHost
dwm
explorer 10.0.14393.0 (rs1_release.160715-1616) 10.0.14393.0 Microsoft Corporation
hkcmd 8.15.10.2900 8.15.10.2900 Intel Corporation
Idle
igfxpers 8.15.10.2900 8.15.10.2900 Intel Corporation
lsass
MBAMService
mDNSResponder
Memory Compression
powershell_ise 10.0.14393.103 (rs1_release_inmarket.160819-1924) 10.0.14393.103 Microsoft Corporation
RuntimeBroker 10.0.14393.0 (rs1_release.160715-1616) 10.0.14393.0 Microsoft Corporation
SearchFilterHost
SearchIndexer
SearchProtocolHost
SearchUI 10.0.14393.953 (rs1_release_inmarket.170303-1614) 10.0.14393.953 Microsoft Corporation
services
ShellExperienceHost 10.0.14393.447 (rs1_release_inmarket.161102-0100) 10.0.14393.447 Microsoft Corporation
sihost 10.0.14393.0 (rs1_release.160715-1616) 10.0.14393.0 Microsoft Corporation
smss
spoolsv
sqlwriter
svchost
svchost
svchost
svchost
svchost
svchost
svchost
svchost
svchost
svchost
svchost
svchost
svchost
svchost
svchost
svchost 10.0.14393.0 (rs1_release.160715-1616) 10.0.14393.0 Microsoft Corporation
System
taskhostw 10.0.14393.0 (rs1_release.160715-1616) 10.0.14393.0 Microsoft Corporation
ToolbarUpdater
wininit
winlogon
WtuSystemSupport
WUDFHost
------------ END PROCESS LIST ------------
Users and Groups:
When I call getUsersAndGroups before getRunningProcesses the output of getUsersAndGroups is injected in getRunningProcesses and worse, no running processes are listed at all, but rather a lot of blank lines.
How can I separate or control the output of getUsersAndGroups so that it outputs before the output of getRunningProcesses?
The output of the injected output looks like this:
Running Processes:
------------ START PROCESS LIST ------------
Username Group
-------- -----
Administrator Administrators
debug255 Administrators;Hyper-V Administrators;Performance Log Users
DefaultAccount System Managed Accounts Group
Guest Guests
------------ END PROCESS LIST ------------
Thank you so much for your help!
tl; dr:
The underlying problem affects both Windows PowerShell and PowerShell (Core) 7+, up to at least v7.3.1, and, since it is a(n unfortunate) side effect of by-design behavior, may or may not get fixed.
To prevent output from appearing out of order, force synchronous display output, by explicitly calling Format-Table or Out-Host:
getUsersAndGroups | Format-Table
getRunningProcesses | Format-Table
Both Format-Table and Out-Host fix what is primarily a display problem, but they are suboptimal solutions in that they both interfere with providing in the output as data:
Format-Table outputs formatting instructions instead of data, which are only meaningful to PowerShell's for-display output-formatting system, namely when the output goes to the display or to one of the Out-* cmdlets, notably including Out-File and therefore also >. The resulting format is not suitable for programmatic processing.
Out-Host outputs no data at all and prints directly to the display, with no ability to capture or redirect it.
Relevant GitHub issues:
GitHub issue #4594: discussion of the surprising asynchronous behavior in general.
GitHub issue #13985: potential data loss when using the CLI.
Background information:
Inside a PowerShell session:
This is primarily a display problem, and you do not need this workaround for capturing output in a variable, redirecting it to a file, or passing it on through the pipeline.
You do need it for interactive scripts that rely on display output to show in output order, which notably includes ensuring that relevant information prints before an interactive prompt is presented; e.g.:
# !! Without Format-table, the prompt shows *first*.
[pscustomobject] #{ foo = 1; bar = 2 } | Format-Table
Read-Host 'Does the above look OK?'
From the outside, when calling the PowerShell CLI (powershell -file ... or powershell -command ...):
Actual data loss may occur if Out-Host is not used, because pending asynchronous output may never get to print if the script / command ends with exit - see GitHub issue #13985; e.g.:
# !! Prints only 'first'
powershell.exe -command "'first'; [pscustomobject] #{ foo = 'bar' }; exit"
However, unlike in intra-PowerShell-session use, Format-Table or Out-Host fix both the display and the data-capturing / redirection problem, because even Out-Host's output is sent to stdout, as seen by an outside caller (but note that the for-display representations that PowerShell's output-formatting system produces aren't generally suitable for programmatic processing).[1]
Note: All of the above equally applies to PowerShell (Core) 7+ and its pwsh CLI, up to at least v7.3.1.
The explanation of PowerShell's problematic behavior in this case:
It may helpful to demonstrate the problem with an MCVE (Minimal, Complete, and Verifiable Example):
Write-Host "-- before"
[pscustomobject] #{ one = 1; two = 2; three = 3 }
Write-Host "-- after"
In PSv5+, this yields:
-- before
-- after
one two three
--- --- -----
1 2 3
What happened?
The Write-Host calls produced output synchronously.
It is worth noting that Write-Host bypasses the normal success output stream and (in effect) writes directly to the console - mostly, even though there are legitimate uses, Write-Host should be avoided.
However, note that even output objects sent to the success output stream can be displayed synchronously, and frequently are, notably objects that are instances of primitive .NET types, such as strings and numbers, as well as objects whose implicit output formatting results in non-tabular output as well as types that have explicit formatting data associated with them (see below).
The implicit output - from not capturing the output from statement [pscustomobject] #{ one = 1; two = 2; three = 3 } - was unexpectedly not synchronous:
A blank line was initially produced.
All actual output followed the final Write-Host call.
This helpful answer explains why that happens; in short:
Implicit output is formatted based on the type of objects being output; in the case at hand, Format-Table is implicitly used.
In Psv5+, implicitly applied Format-Table now waits up to 300 msecs. in order to determine suitable column widths.
Note, however, that this only applies to output objects for whose type table-formatting instructions are not predefined; if they are, they determine the column widths ahead of time, and no waiting occurs.
To test whether a given type with full name <FullTypeName> has table-formatting data associated with it, you can use the following command:
# Outputs $true, if <FullTypeName> has predefined table-formatting data.
Get-FormatData <FullTypeName> -PowerShellVersion $PSVersionTable.PSVersion |
Where-Object {
$_.FormatViewDefinition.Control.ForEach('GetType') -contains [System.Management.Automation.TableControl]
}
Unfortunately, that means that subsequent commands execute inside that time window and may produce unrelated output (via pipeline-bypassing output commands such as Write-Host) or prompt for user input before Format-Table output starts.
When the PowerShell CLI is called from the outside and exit is called inside the time window, all pending output - including subsequent synchronous output - is effectively discarded.
The problematic behavior is discussed in GitHub issue #4594; while there's still hope for a solution, there has been no activity in a long time.
Note: This answer originally incorrectly "blamed" the PSv5+ 300 msec. delay for potentially surprising standard output formatting behavior (namely that the first object sent to a pipeline determines the display format for all objects in the pipeline, if table formatting is applied - see this answer).
[1] The CLI allows you to request output in a structured text format, namely the XML-based serialization format known as CLIXML, with -OutputFormat Xml. PowerShell uses this format behind the scenes for serializing data across processes, and it is not usually known to outside programs, which is why -OutputFormat Xml is rarely used in practice. Note that when you do use it, the Format-Table / Out-Host workarounds would again not be effective, given that the original output objects are lost.
I'm evaluating JProfiler to see if it can be used in a automated fashion. I'd like to run the tool in offline mode, save a snapshot, export the data, and parse it to see if there is performance degradation since the last run.
I was able to use the jpexport command to export the TelemetryHeap view from a snapshot file to a csv file (sample below). When I look at the csv file I see 10 to 13 columns of data but only 4 headings. Is there documentation that explains the output more fully?
Sample output:
"Time [s]","Committed size","Free size","Used size"
0.0,450,880,000,371,600,000,79,280,000
1.0,450,880,000,371,600,000,79,280,000
2.0,450,880,000,371,600,000,79,280,000
3.0,450,880,000,371,600,000,79,280,000
4.0,450,880,000,371,600,000,79,280,000
5.0,450,880,000,371,600,000,79,280,000
6.0,450,880,000,371,600,000,79,280,000
7.0,450,880,000,355,932,992,94,947,000
8.0,450,880,000,355,932,992,94,947,000
9.58,969,216,000,634,564,992,334,651,008
11.05,1,419,456,000,743,606,016,675,849,984
12.05,1,609,792,000,377,251,008,1,232,541,056
17.33,2,524,032,000,1,115,268,992,1,408,763,008
19.43,2,588,224,000,953,451,008,1,634,772,992
26.08,3,711,936,000,1,547,981,056,2,163,954,944
39.75,3,711,936,000,1,145,185,024,2,566,750,976
40.75,3,711,936,000,1,137,052,032,2,574,884,096
41.75,3,711,936,000,1,137,052,032,2,574,884,096
42.75,3,711,936,000,1,137,052,032,2,574,884,096
43.75,3,711,936,000,1,137,051,008,2,574,885,120
44.75,3,711,936,000,1,137,051,008,2,574,885,120
45.75,3,711,936,000,1,137,051,008,2,574,885,120
46.75,3,711,936,000,1,137,051,008,2,574,885,120
47.75,3,711,936,000,1,137,051,008,2,574,885,120
48.75,3,711,936,000,1,137,051,008,2,574,885,120
49.75,3,711,936,000,1,137,051,008,2,574,885,120
50.75,3,711,936,000,1,137,051,008,2,574,885,120
51.75,3,711,936,000,1,137,051,008,2,574,885,120
52.75,3,711,936,000,1,137,051,008,2,574,885,120
53.75,3,711,936,000,1,137,051,008,2,574,885,120
54.75,3,711,936,000,1,137,051,008,2,574,885,120
55.75,3,711,936,000,1,137,051,008,2,574,885,120
56.75,3,711,936,000,1,137,051,008,2,574,885,120
57.75,3,711,936,000,1,137,051,008,2,574,885,120
58.75,3,711,936,000,1,137,051,008,2,574,885,120
60.96,3,711,936,000,1,137,051,008,2,574,885,120
68.73,3,711,936,000,1,137,051,008,2,574,885,120
74.39,3,711,936,000,1,137,051,008,2,574,885,120