Any geniuses out here that could think of a reason why the main MySQL user of our production servers lost it last character?

Im kinda flabbergasted. The username went from username to usernam. And I can't think of any reason why that could happen.

This hurts my brain :)

@Lukas it scares the shit out of me. Nobody logged in during the time of the chance as far as we could find.

This probably going to be a late night, or no night. Not really sure how to explain what just happend.

Somehow all usernames longer then a fixed set of characters got shortend.

Reminiscent of mid-stroke experience

Maybe a changed datatype of the user table...long shot, but the only thing I can think of@aronmik

didnt use MySql much but found this
https://dev.mysql.com/doc/refman/...
any chance u had a version downgrade or had a user longer than max allowed?

@xalez thanks, we checked for system changes but could find nothing. Different usernames changed to a shorter form so cant be the max time.

Only "abnormal" Activity was an export of a useracount from directadmin. But we have been running does for weeks. And not really sure how an export could change usernames. Well other people are going to look into it now. I hope that when I wake up somebody can tell me what happend. And fix that this can never happen again.

@Lukas could check that, but indeed a long shot. Weird thing is that when it happend nobody was active on the server. If somebody on our team did it by accident I would feel a lot better.

Using SSD for storage?

Bit flips?

Is this on ZFS or another filesystem that does checksumming?

Just following to listen to X-Files theme song playing on the background

What's the column type and the table collation? Do you have a backup you can compare to? There could have a been a weird character buffer overflow error on an insert or so in something updated and forced a more strict type. I would backup then go through all the things that touch the DB on the user table and try to make it happen again.

Thanks for all the suggestions.

I get the idea that some people misunderstood me or that I explained it badly.

Its about the MySQL system database. The MySQL documentation says this about it. "You should never alter the structure of tables in the MySQL database in any matter" and we did not. We are not really rebellious so tend to listen to documentation. Defaults are still the same. So no difference in field length or type.

We did not find anything yet. But since we are migrating away from this old server we are watching it closely and hopping it wont happen again.

@icarus its the table for the MySQL users. So it became impossible to connect to the databases. Breaking all products that use a database.

@slar it where several usernames that got shorted. A bit flip wont do that? I have no experience with them so Ik could be wrong.

@adnanh no, we are not.

@conna I have no clue what you're talking about and Google was not much help. Sorry

@aronmik if you have emails let people log in with that. But yeah. Backups.

@icarus Haha, euh maybe I explained it again badly. There was no data lost expect for the last characters of the usernames that are used to connect to the MySQL database. So changing the usernames back in MySQL fixed it.

@aronmik I've actually run in to this before trying to internationalize a site allow utf8.
I see it happen to individual rows at a time. I really think you changed your DB schema somehow. They or someone injected you trying to hack their way in.

@icarus you cannot change that database scheme from the users that are used to log in to the server and serve the apllication.

And adding the characters back fixed the problem. There was no need to change the DB scheme to do that.