8
donuts
9d

I need to estimate how much ram and CPUs my team will need next year for our apps... That have yet to be built.

We load a lot of data feeds with batch processes running on a few large machines, some can use like 30 GB RAM at times...) which should be a lot less if we get the data real time I hope...

But wondering how to estimate well... I sorta did a worse case analysis where I just multiply and sum # CPUs/memory* nodes*approx apps...

Comes out to be like 600 CPUs and 800 RAM... So wondering if that's ok...

RAM is ok but # of CPUs is way higher bc now all the apps basically run on their own machines...

Comments
  • 3
    Answer that stupid question with a stupidly high guesstimate
  • 1
    @Jilano well I mean we have existing apps that sit on a few machines that need to be migrated.

    But these machines are big and the running several apps at once. So if I just multiply the specs by # apps, I guess would be crazy big and actually probably similar to what I just did...

    So just wondering for cloud is that a realistic answer? The savings just come from how the CPUs are actually used?

    Ie. When if I built a container with 30 CPUs, the host server isn't going to allocate all 30 but just make sure there enough in the shared reserve?

    I guess sorta like email storage. U get 10gb but it's not like Google says on our hard drive, this specific 10gb of space is it's whether u use it or not.
  • 4
    You need to have measurements before you can make that estimate.

    You'll also need some sort of computation to go along with the spreadsheet to make the finance guy buy your capacity planning estimate.

    - find your overall max usage of a resource at peak (x)
    - if you currently need more hardware, determine how far off your metric you are for each workload, express as average percentage. Sum percentages, divide by 100, add 1. This is your current operational deficit coefficient (Co)
    - get the percentage growth in customers estimated for the coming year from the business, express as a percentage vs current load.* Repeat percent->coefficient translation for this value. This is your growth coefficient (Cg).

    Your projected resource then becomes:

    Delta r = (x * Co * Cg) - x

    New apps (load) in the coming year will need a new hardware estimate independent of current apps and should be its own discrete computation added to this value. Don't mix the chocolate and the peanut butter.

    *Assuming linearity is fine in most cases for capacity planning unless you have a reason to suspect exponentiation. YoY computations will act as a normalizing logarithm for overestimation in a given year.
  • 4
    Make a stupidly high estimate, then use any spare resources to mine Monero for personal profit.
    Bonus: the cpu and ram usage will make it look like you're actually using those resources, and you can turn off individual miners when you actually need the cpu.
    Ping me for setup assistance.

    ...I'm not even sure I'm joking about this 🤔
  • 0
    What The actual fuck? Increase the capacity dynamically as you go along. It makes much more sense then to buy all the hardware upfront...
  • 2
    @magicMirror my company maintains it's own cloud so they're asking us how much we think we will need for our apps... That don't exist yet... So they can order more hardware I guess... Or not...

    From previous experience they always undershoot... So we get like "sorry U need to wait until we get more hardware in order to move ur app to PROD"
  • 1
    @donuts What?
    How does that make any kind of sense? why not extend to a VPS network over a commercial cloud until you buy new hardware?

    And Why not reduce the Resource requriments for old apps instead? maybe retire old crap, amd release the resources?
  • 0
    @magicMirror

    Retire old crap, that's the point... Eventually

    Not using commercial... Security, data, and "it's cheaper to do in house"

    Let's just say my company is global so very big.

    Not a tech company but if I told you the name your probably know it.
  • 1
    How are you not in a cloud yet?
    AWS spot instances are perfect for this
  • 0
    @dan-pud
    Large companies are being sold on the "private cloud," bc control issues.
  • 1
    @donuts @SortOfTested Someone needs thier job maintaing the hardware...
    someone else want to sell.youbthe hardware. storage, cables, network, etc.

    they should adopt a hybrid solution - buy hardware, use it until capacity, and extend to cloud to handle bursts/new stuff, until they are ready.
  • 2
    @magicMirror
    Not my job to try and get geriatrics to make good decisions on topics they don't understand. Easier to wait for them to die and let it be not my problem.
  • 0
    12 core, 16 gb, done
  • 0
    > From previous experience they

    > always undershoot...

    I see this happen a lot.

    To make a more accurate estimate, we need some figures to go on.

    Also, don't forget communication bandwidth, you might run out of it. :-)

    As a general rule of thumb, whatever you estimate will be wrong, so get double or triple what you think you will need.

    I tend towards double, and then wish I had got triple..
  • 0
    Oh and electricity usage !

    Making sure you have enough actual power from the electricity company to power them all up.

    And, will you be needing an UPS for that too ?

    Or writing your code such that any power cut doesn't cause database errors ?
  • 0
    Also... how to keep things cool. AC is expensive.
Add Comment