5
LLAMS
4y

I ran a big long-running terraform apply and somehow thought it would still work if I locked my laptop.
When I went back the next day (I know lol) terraform was hanging, had to force stop which screwed up the remote tfstate.
Had to spend a whole day manually deleting about 70 AWS resources that terraform created but had no knowledge of because of the corrupted state.

Comments
  • 2
    that's Windows, buddy... It fucks your session up a few minutes after you lock your screen.
  • 1
    Isn't this something you'd want to run remotely somewhere safe? :P
  • 2
    Get thee to a Linux.
  • 1
    Yup! I've done that before. It sucks

    - "terraform import" is awesome
    - Run Terraform in Jenkins on AWS, it's a lifesaver
    - Keep your state in S3 so Jenkins and your local laptop are always sync'd (Terrafrom can do this natively)
    - Avoid SSH provisioning in Terraform at all costs!!! Use cloud-init scripts and pre built AMIs
  • 2
    @devphobe Yeah we do keep state in S3. This was purely me doing a dumb and screwing the apply up so bad the state didnt even get persisted 😅
  • 4
    also this was on macOS not Windows
  • 1
    @LLAMS
    Get it to a server where the workload can be detached and continue working 😊
  • 1
    @LLAMS that is interesting. So Macs do mess up your session too? Damn it!
  • 1
    Actually you can't even configure Macs not to sleep when lid is closed.
Add Comment