Do all the things like ++ or -- rants, post your own rants, comment on others' rants and build your customized dev avatarSign Up
sbiewald352231dThreading in Powershell? im.
Anyway, most of time your thread waits for AWS, right? During this a thread will not do anything, and - besides memory - not consume CPU resources.
Care has to be taken when many of these receive their response at nearly the same time and are suddenly using the CPU, but I doubt a small amount (10 to 20) will let your computer break down.
Besides that, caching is a very good idea as you already wrote.
Do you mind to tell if that was what you wanted to know? While you have written a lot, I'm not sure where your problem is.
@sbiewald Yes thank you! I'm specifically exposing these functions through a web front end so team members can basicaly run my scripts through a simple GET to a url, speeding up the process is the name of the game I'm going for, so I'd like to query nightly and cache, then pull the info from the updated cache.
Each thread creates a huge initial overhead. Reusing threads in a queued way allows you to save some of the cost associated with context switching.
No idea how you'd do that with goddamn PowerShell tho
@kescherRant This is actually a really interesting point you brought up, I didn't think about reusing threads, it looks like the tutorials I found could be improved upon.
So I have created a pool with min/max threads, and I have a loop that iterates theough my AWS data and passes the ARN's into a thread object, it looks like my code is creating a new thread object, passing my scriptblock, and aws data as a param block and adds all of those to an arraylist. At the end of the loop the thread object gets invoked asynchronously. (I think this is so the next thread object can get created before waiting for the thread to process)
then once the loop finishes processing the creation and async invoke of the threads, I have a while loop that is 'true' while there is any thread still processing. Once every thread is in the 'completed' status, I fetch my results by reiterating through the arraylist of threads and collecting the results for processing/formatting.
Also, I'm not using the module PSJobs for this, because I wanted to get an understanding of runspaces (powershell's multithreading)
sbiewald352231dYou know you can include arbitrary C# code in Powershell scripts or call any .NET api? You might want to use lower level functionality in your code.
@sbiewald yup! I've been trying to do that whenever possible to keep things speedy. I found some other really nice little performance boosters here and there too that make some really massive improvements in execution speed.
Like instead of using a regular array
$myVar = @()
use this arraylist instead
$myVar = [system.collections.arraylist]@()
it's soooo much faster to append items to the list. Or this new one I found just last week. There's a small pipeline tax you pay when piping to Out-Null but that can be completely avoided with this: