2

One of team member was showcasing their time series modelling in ML. ARIMA I guess. I remember him saying that the accuracy is 50%.
Isn't that same as a coin toss output? Wouldn't any baseline model require accuracy greater than 50%?

Comments
  • 3
    It all depends on the requirements which might mean that an accuracy of 50% is enough to meet the needs of the project. If I’m running a campaign and the model can possibly determine 50% of potential customers from a given dataset then that would be enough to make a good sale even if the model would be needing further improvement it’s still better than a coin toss.
  • 4
    A "coin toss" would be a classification model that has to classify observations onto two clearly defined classes.
    A model that has to classify stuff amount three or more possible classes and gets it right 50% of the time already has an advantage because, if the odds are the same for each class, each would be less than 50%.

    Now, ARIMA is a regression model, not a classification one. Imagine trying to predict stock prices of a given company in the next one hour within an error margin of 25c. If the stock is currently trading at $10 and we assume that it cannot go below $0 nor above $100, there are 400 possible outcomes in the next hour alone!
    If the model gets the price right 50% of the time, you can get rich.
  • 1
    Also let's not forget that accuracy is not the only metric that may be relevant for a model.

    Recall goes in the opposite direction of accuracy and is often times much more important.
Add Comment