powershell,memory , Memory usage in powershell

Memory usage in powershell


Tag: powershell,memory

I am trying to use Powershell to handle the mundane tasks surrounding running many, many iterations of a chemical model, with slightly differing input files. My powershell script creates a directory, copies the model and input files into the directory, and calls the model, which writes a single output file. Once the model is finished, the script iterates one more time through the loop and does everything again (I have included the script below). My problem is that the memory usage of powershell continually rises, until all the RAM is occupied. Calling the model directly (i.e. simply clicking on the .exe file) produces no such issues, it uses very little memory. What unnecessary information is powershell remembering and how can I force it to keep the memory clear? I'm new to Powershell, so very simple answers, with basically no assumed prior knowledge would be greatly appreciated! Thanks for your help,

#say where the input and the output should be found:


#unique name for this particular set of runs:

$names= '17_exp1' , '17_exp2',  '18_exp2',  '20am_exp1', '20am_exp2' , '20pm_exp1',  '20pm_exp2',  '20pm_exp3',  '21_exp1', '21_exp2',  '21_exp3', '05_exp1', '05_exp2', '07_exp1', '07_exp2', '07_exp3'

#find the bits of name for the directories:
ForEach ($i in $names)

echo $i
if ($i -eq '17_exp1') {$fn='17_11_2013_15_00-15_41_expa1_NH4'}
if ($i -eq '17_exp2') {$fn='17_11_2013_16_59-17_49_expa2_NH4'}
if ($i -eq '18_exp2') {$fn='18_11_2013_04_22-05_30_expa2_NH4'}
if ($i -eq '20am_exp1') {$fn='20_11_2013_02_26-03_19_expa1_NH4'}
if ($i -eq '20am_exp2') {$fn='20_11_2013_08_32-09_15_expa2_NH4'}
if ($i -eq '20pm_exp1') {$fn='20_11_2013_1508-1551_expa1_NH4'}
if ($i -eq '20pm_exp2') {$fn='20_11_2013_2015-2110_expa2_NH4'}
if ($i -eq '20pm_exp3') {$fn='20_11_2013_2240-2335_expa3_NH4'}
if ($i -eq '21_exp1')   {$fn='21_11_2013_03_51-04_58_expa1_NH4'}
if ($i -eq '21_exp2') {$fn='21_11_2013_06_52-07_44_expa2_NH4'}
if ($i -eq '21_exp3') {$fn='21_11_2013_08_57-09_53_expa3_NH4'}
if ($i -eq '05_exp1') {$fn='05_12_2013_11_59-12_40_expa1_NH4'}
if ($i -eq '05_exp2') {$fn='05_12_2013_13_53-14_30_expa2_NH4'}
if ($i -eq '07_exp1') {$fn='07_12_2013_10_04-10_45_expa1_NH4'}
if ($i -eq '07_exp2') {$fn='07_12_2013_11_26-12_06_expa2_NH4'}
if ($i -eq '07_exp3') {$fn='07_12_2013_12_58-13_40_expa3_NH4'}
if ($i -eq '25_pt1_C9_exp1') {$fn='25_09_2014_19_40-20_15_expa1'}
if ($i -eq '25_pt1_C9_exp2') {$fn='25_09_2014_21_00-21_31_expa2'}
if ($i -eq '25_pt1_C9_exp3') {$fn='25_09_2014_22_11-22_41_expa3'}
if ($i -eq '28_pt1_C9_exp1') {$fn='28_09_2014_16_40-17_06_expa1'}
if ($i -eq '28_pt1_C9_exp2') {$fn='28_09_2014_17_44-18_10_expa2'}
if ($i -eq '28_pt1_C9_exp3') {$fn='28_09_2014_18_52-19_15_expa3'}
if ($i -eq '28_pt1_C9_exp4') {$fn='28_09_2014_19_54-20_20_expa4'}
if ($i -eq '24_pt2_C9_exp1') {$fn='24_09_2014_16_22-16_50_expa1'}
if ($i -eq '24_pt2_C9_exp2') {$fn='24_09_2014_17_35-18_00_expa2'}
if ($i -eq '24_pt2_C9_exp3') {$fn='24_09_2014_18_47-19_15_expa3'}
if ($i -eq '25_pt2_C9_exp1') {$fn='25_09_2014_23_45-00_10_expa1'}
if ($i -eq '25_pt2_C9_exp2') {$fn='26_09_2014_01_02-01_30_expa2'}
if ($i -eq '25_pt2_C9_exp3') {$fn='26_09_2014_02_14-02_40_expa3'}
if ($i -eq '28_pt2_C9_exp1') {$fn='28_09_2014_20_55-21_20_expa1'}
if ($i -eq '28_pt2_C9_exp2') {$fn='28_09_2014_22_12-22_41_expa2'}
if ($i -eq '28_pt2_C9_exp3') {$fn='28_09_2014_23_31-23_55_expa3'}

#create the new directory, and copy input and model files into it:
echo $new_dir

new-item $destination\$new_dir -itemtype directory

copy-item $source\work\PSI\CLOUD\Windows_modelling\models\chamber_model_$suffix2".exe" $destination\$new_dir\

copy-item $source\work\PSI\CLOUD\Windows_modelling\expansion_input_files\input_chamber_$fn.dat $destination\$new_dir\input_chamber.dat
copy-item $source\work\PSI\CLOUD\Windows_modelling\libraries_and_stuff\* $destination\$new_dir\
copy-item $source\work\PSI\CLOUD\Windows_modelling\AIM_input\AMS_frac_$fn$suffix.dat $destination\$new_dir\AMS_frac.dat
copy-item $source\work\PSI\CLOUD\Windows_modelling\AIM_input\dist_$fn$suffix.dat $destination\$new_dir\dist.dat
copy-item $source\work\PSI\CLOUD\Windows_modelling\AIM_input\traj_$fn$suffix.dat $destination\$new_dir\traj.dat
copy-item $source\work\PSI\CLOUD\Windows_modelling\AIM_input\*$fn$suffix.dat $destination\$new_dir\

#now go there:
cd $destination\$new_dir
# and call the model, dumping any of the stuff the model outputs to the screen:
) | Out-Null



When you are using the command $(blah.exe) | out-null this causes Powershell to retain all of the output of blah.exe in memory. I tested on my system a one-line script:

cmd /c "dir c:\*.* /s" | out-null

Powershell memory usage does not climb. But with:

$(cmd /c "dir c:\*.* /s") | out-null

Powershell memory starts climbing immediately and keeps climbing.


How to retrieve the name and path of VM's through powercli

am using the below script to retrieve the NAME and PATH of the VM's and in the PATH am getting the full length path which i dont want it, i just need the path which is displaying after the Resources in the output here is my code: function Get-Path{ param($Object)...

Java - allocation in enhanced for loop

I have small problem in java with allocation of my object in "enhanced for loop". I tried something like this, which gives me later nullPointerException: SomeClass tab[] = new SomeClass[100]; for( SomeClass x : tab){ x = new SomeClass(); } So... Earlier I code in C++ and It was possible...

Get list of files whose creation date is greater than some date time

I have a date 2015/05/28 I want to list all files using order by creation date whose creation date is greater than that using PowerShell. How could I do that? I googled for it and found Get-ChildItem "C:\Users\gerhardl\Documents\My Received Files" but no idea how to compare it with creation date...

Loop Issue - Remote Server

I wrote a small script to get some basic information off a few remote servers. But my output it is a bit odd. I believe my issue is with my $DRIVE function. Code: $serversList = 'svr01.xxx.com', 'svr03.xxx.com', 'svr05.xxx.com', 'svr06.xxx.com', 'svr08.xxx.com' #End of Server List Write-Output "Start of Hal0 `n"; ForEach...

Logging actual error when script fails

I have a script here that reads a list of computers and changes the administrator password. When the script is ran, it'll have a log file that says whether the task succeeded or fail. But I also want to log the actual error to the log when it fails. How...

Enhancing the pipeline's content?

Let's assume that you have a command that compresses files using 7-zip that accepts values from the pipeline: Function New-Archive { [CmdletBinding()] param ( [Parameter(Mandatory=$True,ValueFromPipeline=$True,ValueFromPipelineByPropertyName=$True)][Alias('FullName')] [string[]]$Files, [string]$Archive='Archive.zip' ) BEGIN {} PROCESS { Foreach ($File in $Files) { & 7z a -tZIP $Archive $File } } END {} } Assuming the...

Scrapy Memory Error (too many requests) Python 2.7

I've been running a crawler in Scrapy to crawl a large site I'd rather not mention. I use the tutorial spider as a template, then I created a series of starting requests and let it crawl from there, using something like this: def start_requests(self): f = open('zipcodes.csv', 'r') lines =...

Why doesn't “go get gopkg.in/…” work while “go get github.com/…” OK?

I try to use go get gopkg.in/fatih/pool.v2 to install pool according to Readme.md, but can't success: C:\Users\xiaona\Documents\GitHub> go get -v gopkg.in/fatih/pool.v2 Fetching https://gopkg.in/fatih/pool.v2?go-get=1 https fetch failed. Fetching http://gopkg.in/fatih/pool.v2?go-get=1 import "gopkg.in/fatih/pool.v2": http/https fetch: Get http://gopkg.in/fatih/poo l.v2?go-get=1: dial tcp ConnectEx tcp: A connection attempt failed because the connected party did not...

PowerShell logic to remove objects from Array

I'm trying to remove objects from an array that contain duplicates and only keep the ones with the highest number in TasteCode. The example below is highly simplified, but it shows the problem. Example: $Fruits Name | Color | TasteCode ----- ------ --------- Apple | Red | 2 Apple |...

Error with Get-ADUser: Invalid enumeration context

I posted this question the other day Extract e-mail from grouped objects $OuUser = @{} $OuUser = Get-AdUser -Properties * -Filter * -SearchBase "domain" $Duplicates = $OuUser | Select samaccountname, mail,UserPrincipalName | Group-Object Mail | Where{$_.Count -gt 1} $Duplicates | Select Name,Count,@{l='Accounts';e={($_.Group|Select -Expand samaccountname) -join ';'}} | Export-CSV E:\Damo\Duplicates.csv -NoTypeInfo...

CPU usage missing from log for some processes

I'm trying to get information on all processes and services and save it into a .log. Here my Problem: Some processes have noting inside the CPU table, there I want to add something like "Nothing". How can I achieve that? That is my code so far: $ProcessTable = @{Expression={$_.ProcessName};Label="ProcessName";Width=40}, @{Expression={$_.CPU};Label="CPU";Width=20}...

Using --check on a md5sum command generated checksum file is failing

I'm encountering an error when I try and run md5sum -c on a checksum file I generated. I should mention that I'm running this from PowerShell (as a script will be running this eventually) and this is the cygwin version of md5sum. I have a test file, jira_defect.txt and I've...

PowerShell XML formatting issue

I'm having trouble getting a properly formatted XML file using the following code: # Get Computer System info $CompInfo = Get-WmiObject Win32_ComputerSystem $CompName = $CompInfo.Name # Get IP Address $IPAddressTemp = Test-Connection $CompName -count 1 | select Address,Ipv4Address $IPAddress = $IPAddressTemp.IPV4Address # Begin Writing values to XML Document $filePath =...

Get IP address of the Network Adapter of a computer having No gateway

I've a project to add persistent routes on a list of servers which I need to do through Powershell. Our environment has 2 NIC's for every server , one production one backup. For this I need to fetch the Backup IP address of the computer , replace the last octet...

Remove all folders .old

I'm trying to delete all folders in \\kiewitplaza\vdi\Appsense_profiles that end with .old. The piece I have that says Write-Host $name is just me trying to test before I delete anything. $name = Get-ChildItem "\\kiewitplaza\vdi\Appsense_profiles" if ($name.name.EndsWith(".old")) { Write-Host $name #Remove-Item "\\kiewitplaza\vdi\Appsense_profiles\$name.old" } ...

Join SQL query Results and Get-ChildItem Results

Background: I have a directory with a number of files that are imported to SQL server. Task: Creating a PowerShell script which will pick up files within this directory and use the filenames as in the SQL query. Ultimate objective: To display SQL results besides the filenames but the resultset...

Heroku RAM not increasing with upgraded dynos

I have a massive function i have been calling manually through the heroku rails console. I have been receiving the error rapid fire in my logs: 2015-06-22T14:56:42.940517+00:00 heroku[run.9877]: Process running mem=575M(112.4%) 2015-06-22T14:56:42.940517+00:00 heroku[run.9877]: Error R14 (Memory quota exceeded) A 1X dyno is suppose to have 512 MB of RAM. I...

Increment Serial Number using EXIF

I am using ExifTool to change the camera body serial number to be a unique serial number for each image in a group of images numbering several hundred. The camera body serial number is being used as a second place, in addition to where the serial number for the image...

Send email with body consisting of objects

I am trying to write a script to search through AD and look for any stale computers and then sends an email to me. However, there are 3 requirements: get the lastlogondate for any pc that is older than 90 days once I get that list, I do a ping...

Turn environment variable into an array

I need to pass an array to a PowerShell subprocess and was wondering how I can turn an environment variable (string) into a PowerShell array. Is there a convention I need to follow so PowerShell will do it for me automatically or I just need to parse it myself? I'm...

Post messages from async threads to main thread in F#

There is a subscription to an observable that sends out log messages. Some of the log messages come from other threads because they are are in F# async blocks. I need to be able to write out the messages from the main thread. Here is the code that currently filters...

Format a command in powershell including a comma, can't find the right way to escape

I have a command that I need to run in a Powershell script, the command is : ".\pacli DELETEUSER DESTUSER='"[email protected]`,com"' sessionid=333" | invoke-expression The comma (,) in here :[email protected],com is not a mistake, and that's what is giving me the hard time. I tried to escape the comma with `...

Define an array with prefixes using the range operator

From this I can see I can do this with range operator: $myArray = @(4..9) But how can I do something like this: $myArray = @(usr4..9) I have tried various deviations of this with nonsense errors: $myArray = 4..9 | %\{ "usr$_"\} ...

Color a cell on the basis of another cell value

I have a file output.txt having below content: SERVER_NAME MOUNT_POINT TOTAL_SPACE USED_SPACE USED_PERCENTAGE AVAILABLE_SPACE Server_1 /dev 1200G 537G 54% 464G Server_2 /dev 600G 490G 85% 94G Server_3 /dev 600G 402G 69% 181G Backup_server /storage 800G 682G 72% 278G Using following script, I formatted my output.txt file to get a notification...

Powershell workflow - Get-Service not filtering

I am trying to restart Windows services in parallel via Powershell workflow. When I use Get-Service -DisplayName "S*" outside of workflow I get expected results. However, when I use the same in workflow, I get ALL services instead. It seems that -Displayname parameter is ignored in workflow. How do I...

Search for certain UPN suffix

How do I search AD users by UPN suffix in AD? The users are under Uofguelph/SEC213/users OU. Some users have @sec213.com suffix and some have @home.sec213.com suffix. I need to search for users who only have @sec213.com suffix. I have the following two code snippets. Code 1 gets all accounts...

Powershell Reading text file

I'm just wondering if it's possible to read the content of text file with specific index? What I mean is like this, for example: I have text file like this, 'test1.txt' 12345678900 ## ## readthistext 54321123440 ## ## hellothistext I just want to read the content of text file after...

Memory javascript dictionary

If I add a value to the 1000th element of a javascript dictionary then is there any difference to adding that value to the 0th element assuming those positions are open. I'm speaking in terms of memory consumption. ex. arr[1000] = 'asdf'; versus arr[0] = 'asdf'; ...

How do I write a loop to read text file and insert it to the database

I'm writing a powershell script to load text files from a specific folder into SQL server database, I have manage to do so but only one file by one file. I want the program to do this automatically once the user inputs the path and the file extension. For example:...

How to create a powershell script that triggers a NuGet Update-Package –reinstall?

I have a nuget spec that installs both library file (dll) and two zip packages on a specific folder. Updating the packages is no issue when doing it from Visual Studio 2013 manually. However, when it is being triggered from either our CI servers TeamCity and CCNet, it is only...

Programmatically accessing TFS history [closed]

I have a solution under TFS source control and had written some sample code using an external library that was deleted a few months ago. I am certain that the code was checked in before that class file was deleted. I am able to manually browse to previous versions of...

Extract e-mail from grouped objects

I have some code that pull user accounts from an OU in AD. I am sorting it using Group-Object so I know how many users are using the same email address. Code below, the $OuUser uses Get-ADUser with Properties and Filter set to *. $Duplicates = $OUUser | Select samaccountname,...

How to pass a switch variable?

& "$THIS_SCRIPTS_DIRECTORY_PATH\New-NuGetPackage.ps1" -PushOptions "$pushOptions" ` -Verbose -ProjectFilePath $project -PO "$packOptions" -NPFPPTNG So if I provide the command line above in PowerShell the call works correctly. If I try something like this: if ($NoPromptForPushPackageToNuGetGallery) { $xtraOptions += " -NPFPPTNG " } & "$THIS_SCRIPTS_DIRECTORY_PATH\New-NuGetPackage.ps1" -PushOptions "$pushOptions" ` -Verbose -ProjectFilePath $project -PO "$packOptions"...

Is it safe to read and write on an array of 32 bit data byte by byte?

So I have a void * data of 32 bit unsigned integers which represents the pixels. Is it okay for me to access one of the pixels with a char * and modify the values directly? Or is it better to store my new pixel in a temporary uint32_t variable...

C++11 Allocation Requirement on Strings

I had heard that C++11 was going to require strings to be allocated in contiguous memory. I even thought I saw a stack overflow question on it, but I can't seem to find it. I know that in practice both gcc and Visual Studio do allocate strings contiguously, I'm just...

setting up azure ad certificate auth using powershell

I am attempting to setup an Azure AD application to authenticate using a certificate. I have been using this guide to successfully set this up after failing following this guide. I wish to now automate the process so it can be added to our general build scripts, looking at the...

fread(), solaris to unix portability and use of uninitialised values

Valgrind found the following error and I, after reading the documentation, the code and other questions in here couldn't figure it out why. Valgrind: first warning ~$ valgrind --vgdb=yes --vgdb-error=0 --read-var-info=yes --leak-check=yes --track-origins=yes debitadmin* debitadmin ==20720== Conditional jump or move depends on uninitialised value(s) ==20720== at 0x4013BC6: initialise (dbg.c:199) ==20720==...

Issue filtering out certain event logs from output

So I've coded a little script that sends me the last 24 hours event logs for my Servers in a nice HTML Email to save me having to log on to each Server, etc. It works well except some of our applications produce loads of events that I want to...

SetProcessWorkingSetSize does not work in compiling 64bit

I use the following command to reduce the memory usage of my program, I'm actually testing it, only when I compile the program in 64bit this command does not work fot, no error occurs, only the memory in task managerIt does not decrease too, since in compiling 32bit works perfectly,...

PowerShell Where-Object $_.name -like -in $list

New to PowerShell and ran into a bit of a road block. I am trying to pull program name and version information from multiple servers. I have a list of the program names in a $list variable but the program names also contain the version numbers in them. I am...

What is the `.` shorthand for in a PowerShell pipeline?

I'm looking over a block of code I've used (sourced from another question) and I haven't been able to figure out what the . in .{process represents in this snippet (comments removed): Get-ItemProperty $path | .{process{ if ($_.DisplayName -and $_.UninstallString) { $_ } }} | Select-Object DisplayName, Publisher, InstallDate, DisplayVersion,...

Powershell comparison of text file

I'm just wondering if it's possible to compare two text file and check if there's a difference between them? What I mean is like this, For example I have text1.txt : 123 456 789 000 and I have text2.txt : 123,test,input,bake 789,input,cake,bun Expected output : 456 does not exist in...

Why does piping Get-PSSession to Exit-PSSession not work?

I'm creating a new Powershell session like so: New-PSSession -ConnectionUri https://myserver:12345 -Credential myserver\username -Authentication default This is working fine, I can do Get-PSSession and it provides me with that one session. However, when trying to pipe the result to Exit-PSSession things go haywire. So this: Get-PSSession | Exit-PSSession Or this:...

Good approach to clean application data programmmatically in Android

When I use RecyclerView and/or Picasso. It seems to cache and store alot of data in my application. However, I want to delete the stored data and the cache in a simple way whenever I am exiting my app. What is the simplest way to delete all data stored in...

How do I select a string from a string and replace it in powershell?

I'm trying to write a powershell instance that finds and replaces each instance of text and replaces it. UserRights "rights_wo_view" I need to place parenthesis around the quotes. I've been trying all kinds of things, but I'm running flat. $files = get-item C:\Users\programmer\Documents\Project\tsbrick\*.asp foreach ($file in $files) {(Get-Content $file) select-string...

Filter and delete Registry values with Where-Object

I'm not sure why I'm finding this so difficult. From under a particular registry key, I'd like to query certain registry-values based on their data (not name), and delete the resulting registry-values. For example: How would I delete any values in this key that contain, let's say, "foo". I can...

Program Runs Fine for Hours and Eventually Seg Faults with Memory Address 0x10 [closed]

So I've been tracking some memory issue for the longest time. I'm coding in C++ and I can see that my program mostly works. I am monitoring my resources and I dont think I have a memory leak because my memory used stays below 12% (I'm on a system with...

Unable to increase PHP Memory Limit

I am running into an issue with a project I am working with on my localhost, where I am unable to increase the PHP memory_limit setting. I've tried increasing it directly in the php.ini config: memory_limit = 1024M I've tried increasing it in the projects .htaccess file: php_value memory_limit 1024M...

PS pipe WorkingSet as variable

I have this which gives me the WorkingSet memory usage I'm after: $mem=Get-Process spiceworks | Select-Object Name,@{Name='WorkingSet';Expression={($_.WorkingSet/1KB)}} But I'm looking to only have the actual amount as the variable so I can then work with that to monitor it. I want to set it so it alerts me if it...

PowerShell - Convert CSV to XLSX

I found this script on here, please help me understand this better. What is the whole "gps" part of the script for? What part of this script is overwriting my original CSV and copying in the PC's taskmanager list? Processes. What does this do, why 51? ($xlOut,51) I went searching...