Do all the things like ++ or -- rants, post your own rants, comment on others' rants and build your customized dev avatarSign Up
Stop using excel ffs
They didn't want to set the field format?
You'd need to set the format, fix it to every cell - requiring you to know how many cells are affected.
And here lies the reason why it affects so many people.
Most of these tables are very large.
You don't want to check 1024+ x N rows / cols.
I'm guilty of it, too. Not for genetic data, but sometimes I'm staring at the screen for half n hour to figure out why the calculation didn't work and realized that the Copy n Paste went horribly wrong.
@N00bPancakes @Lor-inc half true.
But you'll need to know the colums before... And in a nutshell - setting a column is essentially setting Cell ColumnName_1:CellColumnName_Max.
And even doing that.... Might not work as intended. Usually you need to disable Proofing - Auto Correct Options. Since pasting large rows and texts triggers AutoCorrect, which is an nefarious lil shit hole.
Access or other databases isn't an alternative solution for Excel.
Excel is an database per se, just with unstructured data.
And things like PowerDrill or other functions are really useful, although shitty.
So my question remains. What is an alternative to Excel?
I use Excel quite alot.
Mostly when I need to transform data.
Yes, I could write an whole Importer / Exporter for it. But when you are used to Excel, writing the Importer / Exporter takes more time (and is even more brittle) then just hammering the data in Excel, apply a few functions and be done.
Since PowerDrill exists, I usually have a few semi exporters of monitoring APIs that generate JSON.
I pump that JSON into a database when it's necessary, but in most cases Excel and PowerDrill is enough to get the job done.
And the Import / Export part is just a tiny lil thing.
Project management, statistics, time calculation and so on.
@SortOfTested thanks, I'll take a look at that. It sounds interesting.
Yes, Jupyter is one of the reasons Python is quite pervasive in bio sciences & gentech.
This whole Excel thing surprises me -- I worked at a biotech firm, and for bioinformatics it was iPython (Jupyter), or GTFO.
Not being a dev was no excuse, everyone had to learn the basics of Python, Numpy and Pandas -- large datasets were either loaded into the main postgres DB, with experimental data sometimes just stored as literals within the notebook or imported as CSV.
The big advantage is that you're not limited to excel tooling this way, but have all these Python visualization tools.
Clustergrammer is for example used a lot for visualizing gene expression:
A nice notebook based on US census data:
I like this example as well:
@bittersweet didn't know that...
Haven't looked / heard from Jupyter yet.
I think quite a lot of companies, especially in research / university / education are "bound" to MS.
I tried to avoid Excel and MS software for a long time.... But at some point it was just impossible. Interaction with clients, presentation, aggregating and evaluating data, pleasing the upper level with nice graphics.
So my relationship is a hate / love thing. When there would be a true substitute, I'd be happy.
I'll try every year to use LibreOffice for a few days... It usually ends in aggression, hate and anger.
Quite a lot of the stuff I use - eg monitoring / data aggregation / representation is based on hand written crawlers.
To deal with different software versions... Dead software that shouldn't be used at all... Servers who are not reachable in a certain way (by the power of VPN and 5 SSH tunnels, I summon the mother of all sockets...).
The thing is: I can usually not change / migrate the software. I cannot "buy" software just because I need it.
I have to be sometimes very creative in that part...
And then I'll just need Excel as a non structured, non complaining database solution to not waste more time, but get the job done. :/
I'd be really happy if there were alternatives, but so far. Nope.
@IntrusionCM Well, the question is indeed "how much can you let go of the spreadsheet model"
Spreadsheets are kind of dumb. If your data has more than 2 dimensions, they don't work that well. If your data is spatial/geographical, they don't work that well. If your data is structured like a (undirected/directed/weighted) graph, spreadsheets don't work well.
Spreadsheets are very limiting.
Excel is usually one of those tools you need because of third parties — but in many fields it's not the best, most powerful or most flexible tool for the job.
If you're tied to Excel, I'd recommend just using xlsx files as your "storage" in Jupyter through openpyxl. That way you can experiment and open up new ways of playing with data, while keeping the compatibility.
Please, let the weather cool down first.
I was shocked two hours ago when someone apologized for contacting me on a friday evening...
It's thursday........ Wait. It's really friday. What the frigging fuck Oo
And even if I Like Jupyter...
It would take a ton of time to migrate the workflows. But I'll reallly take a look at it ;)