Join devRant
Do all the things like
++ or -- rants, post your own rants, comment on others' rants and build your customized dev avatar
Sign Up
Pipeless API
From the creators of devRant, Pipeless lets you power real-time personalized recommendations and activity feeds using a simple API
Learn More
Search - "#multithreading"
-
Got this from a recruiter:
We are looking for a **Senior Android Developer/Lead** at Philadelphia PA
Hiring Mode: Contract
Must have skills:
· 10-12 years mobile experience in developing Android applications
· Solid understanding of Android SDK on frameworks such as: UIKit, CoreData, CoreFoundation, Network Programming, etc.
· Good Knowledge on REST Ful API and JSON Parsing
· Good knowledge on multi-threaded environment and grand central dispatch
· Advanced object-oriented programming and knowledge of design patterns
· Ability to write clean, well-documented, object-oriented code
· Ability to work independently
· Experience with Agile Driven Development
· Up to date with the latest mobile technology and development trends
· Passion for software development- embracing every challenge with a drive to solve it
· Engaging communication skills
My response:
I am terribly sorry but I am completely not interested in working for anyone who might think that this is a job description for an Android engineer.
1. Android was released in September 2008 so finding anyone with 10 years experience now would have to be a Google engineer.
2. UIKit, CoreData, CoreFoundation are all iOS frameworks
3. Grand Central Dispatch is an iOS mechanism for multithreading and is not in Android
4. There are JSON parsing frameworks, no one does that by hand anymore
Please delete me from your emailing list.49 -
In the age of multithreading, parallel processing, and non-blocking IO, you still get messages like this.14
-
I asked my teammates if it would be ok if I made multithreading code for the collisions of our game engine and they just sent me this, I'm going to try anyway. Yolo.2
-
Do you know what's more fun than debugging multithreaded code?
Debugging networked, multithreaded code...3 -
rant
The Java course at our Uni requires us to do an end semester project - A Java App with Swing for GUI and some Multithreading code in it.
They asked us to upload the code to drive. I was bored and was checking out my friends' projects.
The code below is what I saw in one of the projects. They have simply called a thread with an empty run method because the project required to use multithreading concepts, wtf.
But then, It is no surprise to me cause these are the people who memorize code and vomit code for marks.
I am worried that people are going to be awarded degrees and called software engineers.
God save the software industry!24 -
2/2
Multithreading has a new perspective now.
Also, fast is cute.
Part 2 of the awesome change log. 😂6 -
Every single group chat I'm in has started talking about multiple things at once and I'm always SOOOOOOOO tempted to make a comment about multithreading but then I remember that nobody else will find it funny. I NEED MORE PROGRAMMER FRIENDS.4
-
Some people, when confronted with a problem, think: "I know, I'll use multithreading".
Nowthhw tpe yawre o oblems.3 -
Senior Engineer -
Hey. I have a code that hits API to get details and multithreading is implemented. Can you just change the URL formed to hit Api?
Me
Yeah sure why not.
Me
After some time I discover that the initial code itself wasn't working 😐
I realise i need to fix code, fix multithreading and then make the URL changes.
Just finished......Realised had to rant....1 -
CEO - So... We'll have a new side project, it's a small thing, I spoke with a guy, he needs just a small thing...
Me - Ok....... So, what do you need me for?
CEO - Not much, this guy started a project but wants your help on a small part, and you already did something similar for us, so it should be fast, just copy and paste and change a little bit. You probably know better than me.
Me - *Sigh* ... So `friend`, what do you need from me?
Friend - So, I made a crawler that is storing some information on a local file, I just need to add multiprocessing and multithreading, a producer-consumer system with a queue so I can automatically add new links every now and then, a failback system, so when a process doesn't finish, it should be re-queued and crawled later, store all the information on a database cluster (that is not set up), [......]
Me - And when is this supposed to be up and running?
Friend - Your CEO told me you could do this by the weekend. Can we finish this by Friday?
Me - *facepalm* FML13 -
Its such a elating feeling when you implement multithreading perfectly.
I had a task where I read data from one table, searched on it and added the searched data to another table. The table had 0.22 million records.
On single thread it took entire night to do half of the job.
I divided it into 8 thread jobs, and pushed my quadcore to 100% and voila, its done in 2.5Hrs.
Just like Hannibal would have said, "I love it when a plan comes together"5 -
Okay, story time.
Back during 2016, I decided to do a little experiment to test the viability of multithreading in a JavaScript server stack, and I'm not talking about the Node.js way of queuing I/O on background threads, or about WebWorkers that box and convert your arguments to JSON and back during a simple call across two JS contexts.
I'm talking about JavaScript code running concurrently on all cores. I'm talking about replacing the god-awful single-threaded event loop of ECMAScript – the biggest bottleneck in software history – with an honest-to-god, lock-free thread-pool scheduler that executes JS code in parallel, on all cores.
I'm talking about concurrent access to shared mutable state – a big, rightfully-hated mess when done badly – in JavaScript.
This rant is about the many mistakes I made at the time, specifically the biggest – but not the first – of which: publishing some preliminary results very early on.
Every time I showed my work to a JavaScript developer, I'd get negative feedback. Like, unjustified hatred and immediate denial, or outright rejection of the entire concept. Some were even adamantly trying to discourage me from this project.
So I posted a sarcastic question to the Software Engineering Stack Exchange, which was originally worded differently to reflect my frustration, but was later edited by mods to be more serious.
You can see the responses for yourself here: https://goo.gl/poHKpK
Most of the serious answers were along the lines of "multithreading is hard". The top voted response started with this statement: "1) Multithreading is extremely hard, and unfortunately the way you've presented this idea so far implies you're severely underestimating how hard it is."
While I'll admit that my presentation was initially lacking, I later made an entire page to explain the synchronisation mechanism in place, and you can read more about it here, if you're interested:
http://nexusjs.com/architecture/
But what really shocked me was that I had never understood the mindset that all the naysayers adopted until I read that response.
Because the bottom-line of that entire response is an argument: an argument against change.
The average JavaScript developer doesn't want a multithreaded server platform for JavaScript because it means a change of the status quo.
And this is exactly why I started this project. I wanted a highly performant JavaScript platform for servers that's more suitable for real-time applications like transcoding, video streaming, and machine learning.
Nexus does not and will not hold your hand. It will not repeat Node's mistakes and give you nice ways to shoot yourself in the foot later, like `process.on('uncaughtException', ...)` for a catch-all global error handling solution.
No, an uncaught exception will be dealt with like any other self-respecting language: by not ignoring the problem and pretending it doesn't exist. If you write bad code, your program will crash, and you can't rectify a bug in your code by ignoring its presence entirely and using duct tape to scrape something together.
Back on the topic of multithreading, though. Multithreading is known to be hard, that's true. But how do you deal with a difficult solution? You simplify it and break it down, not just disregard it completely; because multithreading has its great advantages, too.
Like, how about we talk performance?
How about distributed algorithms that don't waste 40% of their computing power on agent communication and pointless overhead (like the serialisation/deserialisation of messages across the execution boundary for every single call)?
How about vertical scaling without forking the entire address space (and thus multiplying your application's memory consumption by the number of cores you wish to use)?
How about utilising logical CPUs to the fullest extent, and allowing them to execute JavaScript? Something that isn't even possible with the current model implemented by Node?
Some will say that the performance gains aren't worth the risk. That the possibility of race conditions and deadlocks aren't worth it.
That's the point of cooperative multithreading. It is a way to smartly work around these issues.
If you use promises, they will execute in parallel, to the best of the scheduler's abilities, and if you chain them then they will run consecutively as planned according to their dependency graph.
If your code doesn't access global variables or shared closure variables, or your promises only deal with their provided inputs without side-effects, then no contention will *ever* occur.
If you only read and never modify globals, no contention will ever occur.
Are you seeing the same trend I'm seeing?
Good JavaScript programming practices miraculously coincide with the best practices of thread-safety.
When someone says we shouldn't use multithreading because it's hard, do you know what I like to say to that?
"To multithread, you need a pair."18 -
TIL that TI has no goddamn chill
Texas Instruments released the TI-83+ calculator model in 1996. The Z80 was not at all stock and has the following features:
- 3 access levels (priveleged kernel, kernel, user)
- Locking Flash (R/O when locked for most pages, some pages protected and unreadable as well, only unlockable from protected Flash pages by reading a certain order of bits then setting a port)
- Locking hardware ports (lock state always the same as flash)
- Customizable execution whitelist range (via locked ports)
- Configurable hardware (Flash/RAM size changeable in software via locked ports, max RAM is 8MB which is fucking mental compared to the 64k in the thing)
- Userland virtualization (always-on)
- Reset on violation of security model
- Multithreading
- Software-overclockable CPU
- Hardware MD5 and cert handling
TI made a calculator in 1996 with security features PCs wouldn't see until like 2010 what the *actual* fuck10 -
As a human being, you're multithreaded. You can type with multiple fingers, you can drive and hold a conversation at the same time. The only blocking function we have to deal with is sneezing, where all current activity must be suspended for the duration of the sneeze. That's pretty annoying, especially when you're driving and trying to hold a conversation. You don't want to write code that's sneezy.8
-
That would probably be implementing multithreading in shell scripts.
https://gitlab.com/netikras/bthread
The idea (though not the project itself) was born back when I still was a sysadmin. Maintaining 30k servers 24/7 was quite something for a team of merely ~14 people. That includes 1st line support as well.
So I built a script to automate most of my BAU chores. You could feed a list of servers - tens or hundreds or more - and execute the same action on each of them (actions could be custom or predefined in the list of templates). Neither Puppet nor Chef or Ansible or anything of sorts was consistently deployed in that zoo, not to mention the corp processes made use of those tools even a slower approach than the manual one, so I needed my own solution.
The problem was the timing. I needed all those commands to execute on all the servers. However, as you might expect, some servers could be frozen, others could be in DMZ, some could be long decommed (and not removed from the listings), etc. And these buggars would cause my solution to freeze for longer than I'd like. Not to mention that running something like `sar -q 1 10` on 200 servers is quite time-consuming itself :)
And how do I get that output neatly and consistently (not something you'd easily get with moving the task to a background with '&'. And even with that you would not know when are all the iterations complete!)?
So many challenges...
I started building the threading solution that would
- execute all the tasks in parallel
- do not write anything to disks
- assign a title to each of the tasks
- wait for all the tasks to complete in either
> the same sequence as started
> as soon as the task finishes
- keep track of each task's
> return code
> output
> command
> sequence ID
> title
- execute post-finish actions (e.g. print to the console) for each of the tasks -- all the tracked properties are to be accessible by the post-finish actions.
The biggest challenges were:
a) how do I collect all that output without trashing my filesystems?
b) how do I synchronize all those tasks
c) how do I make the inception possible (threads creating threads that create their own threads and so on).
Took me some time, but I finally got there and created the libbthread library. It utilizes file descriptors, subshells and some piping magic to concentrate the output while keeping track of all the tasks' properties. I now use it extensively in my new tools - the ones where I can't use already existing tools and can't use higher-level languages.4 -
First rant: but I'm so triggered and everyone needs a break from all the EU and PC rants.
It's time to defend JavaScript. That's right, the best frikin language in the universe.
Features:
incredible async code (await/async)
universal support on almost everything connected to the internet
runs on almost all platforms including natively
dynamically interpreted but also internally compiled (like Perl)
gave birth to JSON (you're welcome ppl who remember that the X in AJAX stood for XML)
All these people ranting about JS don't understand that JS isn't frikin magic. It does what it needs to do well.
If you're using it for compute-heavy machine learning, or to maintain a 100k LOC project without Typescript, then why'd you shoot yourself in the foot?
As a proud JS developer I gotta scroll through all these posts gushing over the other languages. Why does nobody rant about using Python for bitcoin mining or Erlang to create a media player?
Cuz if you use the wrong tool for the right job, it's of course gonna blow up in your face.
For example, there was a post claiming JS developers were "scared" of multithreading and only stick in their comfort zone. Like WTF when NodeJS came out everything was multithreaded. It took some brave developers to step out of the comfort zone to embrace the event loop.
For a web app, things like PHP and Node should only be doing light transforms between the database information and HTML anyways. You get one thread to handle the server because you're keeping other threads open to interface with databases and the filesystem. The Nexus.js dev ranting on all us JS devs and doesn't realize that nobody's actual web server is CPU bound because of writing HTML bodies, thats why we only use 1 thread. We use other worker threads to do the heavy lifting (yes there is a C++ bridge look it up)
Anyways TL;DR plz respect JS developers we're people too. ES7 is magic and please don't shit on ES3 or we'll start shitting on the Python 2-3 conversion (need to maintain an outdated binary just cuz people leave out ()'s in their print statements)
Or at least agree that VB.NET is an abomination and insult to the beauty that is TI-84 BASIC13 -
lesson of the day:
if you enter the office toilet while chewing gum, the simultaneous smell of shit and jaw movement will lead your brain to think you're eating shit3 -
Many of you who have a Windows computer may be familiar with robocopy, xcopy, or move.
These functions? Programs? Whatever they may be, were interesting to me because they were the first things that got me really into batch scripting in the first place.
What was really interesting to me was how I could run multiples of these scripts at a time.
<storytime>
It was warm Spring day in the year of 2007, and my Science teacher at the time needed a way to get files from the school computer to the hard-drive faster. The amount of time that the computer was suggesting was 2 hours. Far too long for her. I told her I’d build her something that could work faster than that. And so started the program would take up more of my time than the AI I had created back in 2009.
</storytime>
This program would scan the entirety of the computer's file system, and create an xcopy batch file for each of these directories. After parsing these files, it would then run all the batch files at once. Multithreading as it were? Looking back on it, the throughput probably wasn't any better than the default copying program windows already had, but the amount of time that it took was less. Instead of 2 hours to finish the task it took 45 minutes. My thought for justifying this program was that; instead of giving one man to do paperwork split the paperwork among many men. So, while a large file is being copied, many smaller files could be copied during that time.
After that day I really couldn't keep my hands off this program. As my knowledge of programming increased, so did my likelihood of editing a piece of the code in this program.
The surmountable amount of updates that this program has gone through is amazing. At version 6.25 it now sits as a standalone batch file. It used to consist of 6 files and however many xcopy batch files that it created for the file migration, now it's just 1 file and dirt simple to run, (well front-end, anyways, the back-end is a masterpiece of weirdness, honestly) it automates adding all the necessary directories and files. Oh, and the name is Latin for Imitate, figured it's a reasonable name for a copying program.
I was 14, so my creativity lacked in the naming department >_<1 -
I've tried multithreading with php, wrote a simple script which checks a series of ip addresses and tries to ftp into them.
I've noticed that the script is running very slow, i checked everything, tested the db, tested my code, i've started to doubt, that my compilation of php was fucked up (btw i did that for the first time).
Then i've started to mesure the time of each db request, but the numbers didn't add up. Then it fucking hit me...
I fucking set the timeout for ftp_connect to 5 seconds, and that was causing the slowdown. I wasted two fucking coding sessions on finding that out.
What a fucking blind moron can I be, holy shit.4 -
Inspired by @Billgates
everyone around is hyped about new tech they get to use, new toys to tinker with, I can see their eyes shining when they hear "let's try and introduce kafka" - they would wiggle their tails all day long if they had ones!
And me? Well, a new potential employer got me so excited I couldn't wash a smile off my face for a few days! You know what they said? "we don't use any frameworks, we focus on clean code, solid, kiss and we write with tdd". Bare java - that's the best position I've heard of in years!
I guess I'm oldschool. But I truly believe their approach is the right one. Not trashing the code with spring [which is turning into smth what systemd is for linux/unix], hibernate and what not.
Just good old java code. Db, multithreading, request-mapping -- all plain, manual and simple.
Amazing!19 -
So I'm writing some multithreaded shit in C that is supposed to work cross-platform. MingW has Posix threads for Windows, so that saved already half of the platform dependency. The other half was that these threads need to run external programs.
Well, there's system(), right? Uhm yes, but it sucks. It's incredibly slow on Windows, and it looks like you can have only one system() call ongoing at the same time. Which kinda defeats the multithreaded driver. Ok, but there's CreateProcessA(), and that doesn't suck.
Fine, now for Linux. The fork/exec hack is quite ugly, but it works and is even fast. Just never use fork() without immediate exec(). First try under Cygwin... crap I fork bombed my system! What is this shit? Ah I fucked up the path names so that the external executable couldn't be run.
Lesson learnt: put an exit() right after the exec() in the path for child process. Should never be reached, but if it goes there, the exit() at least prevents a fork bomb.
Well yeah, sort of works under Cygwin, but only with up to 3 threads. Beyond that, it seems like fork() at some point gives two processes the same PID, and then shit hangs.
Even slapping a mutex around the fork and releasing it only in the parent process didn't help. Fork in Cygwin is like a fork in the ass. posix_spawn() should work better because it can be mapped more easily to the Windows model, but still no dice.
OK, testing under real Linux. Yeah, no issues with that one! But instead, I get some obscure "free(): invalid size" abort. What the fuck would that even mean?! Checking my free() calls: all fine.
Time to fire up GDB in the terminal! Put a catch on the abort signal, mh got just hex data. Shit I forgot to compile with -O0 and -g. Next try. Backtrace shows the full call trace, back to the originating line in my program - which is fclose() on a file.
Ahhh I remember! Under Linux, fclosing a file that is already closed makes the program crash. So probably I was closing it twice. Checking back.. yeah that's where it was.
Shit runs fast on several cores now!8 -
8.5 hours spent trying to figure out a race condition ultimately caused by this line.
There really are no words.11 -
Python sucks so hard!! Slow Interpreter, Broken Multithreading. It really sucks. Overrated language.10
-
Rant:
I am at work, some one says to me this system we are working on is multi threaded. I tell the no its not multi threaded and in this context. Things cannot happen concurrently. Its a single core arm 7tdmi. Arguments ensue abot the difference between multithread multitasking an multiprocessing. I proceed to explain this is a multitasking interrupt driven system. With no context switching or memory segmentation so one heap for all tasks cause thats how we have it configured and there is only one core. So there is no way the error he just described could possibly happen. Then he tells me im wrong but refuses to even look at the processor manual and rejects the Wikipedia entry for multithreading. So I plan on calling off so i can just have the next two weeks off while he trys to figure out why two things ar happening at once on this system. He deserves all the frustration that is to follow.1 -
When I got the current job I started to work on an Android app that a coworker which left the company was doing.
The app was ready at about 40% and was barely usable, it lacked a lot of features and multithreading so with a huge amount of data it used to crash (Android doesn't allow you to make the app freeze for more than 2-3 seconds, it considers that the app is not responding anymore).
After a week or two the work to do was still huge, but one day one of my coworkers came in and ask me if I was able to release a beta for a client the same day... Unexpected deadline.
I spent 8 hour fixing as many bugs as possible and adding multithreading in the most weak parts.
I did it but it was so stressful and the result wasn't even great. In fact I finished the stable version 7 months later.4 -
So I took my old C# project "RotatingCube" for a spin and transformed the unreadable and inefficient mess into a different program, featuring better readability and more comments, with multiple cubes at once, without the shitty flickering.
I did that for school but it was quite fun to tinker with only outputting the differences to a previous output.
Check it out at https://github.com/filthycoding/...!
Next I just need multithreading for performance reasons. -
If there is 2 books you should read before trying to tackle TAOCP... this might be on it.. as well as the Concrete Mathematics book.
Anyway. This book covers not just the fundamentals of modern algorithms and data structures but it also makes the leap to understanding multithreading and algorithms using multithreading.
Some argue the certain concepts in this book are presented without explanation of how they work, but I guess that should be something the reader try’s to figure out from another book or constructive thinking critically. Keeps the reader on their toes for understanding.
This is also the reason many people suggest the sedgewick algorithm books, of which will be posted another day.16 -
Just had an interview, but since I am a smart ass I decide that half an hour notice before the interview is OK after working the whole day with queues, docker & php multithreading, so we start an interview over Skype and my persona was able to duck up how simple joins work as well as function which is supposed to return the sum of even numbers between 2 and provided argument... I was off by 2..
Lesson learned never get in an interview after a whole day of mindfuckery .. Never -
CONTEST - Win big $$$ straight from Wisecrack!
For all those who participated in my original "cracking prime factorization" thread (and several I decided to add just because), I'm offering a whopping $5 to anyone who posts a 64 bit *product* of two primes, which I cant factor. Partly this is a thank you for putting up with me.
FIVE WHOLE DOLLARS! In 1909 money thats $124 dollars! Imagine how many horse and buggy rides you could buy with that back then! Or blowjobs!
Probably not a lot!
But still.
So the contest rules are simple:
Go to
https://asecuritysite.com/encryptio...
Enter 32 for the number of bits per prime, and generate a 64 bit product.
Post it here to enter the contest.
Products must be 64 bits, and the result of just *two* prime numbers. Smaller or larger bit lengths for products won't be accepted at this time.
I'm expecting a few entries on this. Entries will generally be processed in the order of submission, but I reserve the right to wave this rule.
After an entry is accepted, I'll post "challenge accepted. Factoring now."
And from that point on I have no more than 5 hours to factor the number, (but results usually arrive in 30-60 minutes).
If I fail to factor your product in the specified time (from the moment I indicate I've begun factoring), congratulations, you just won $5.
Payment will be made via venmo or other method at my discretion.
One entry per user. Participants from the original thread only, as well as those explicitly mentioned.
Limitations: Factoring shall be be done
1. without *any* table lookup of primes or equivalent measures, 2. using anything greater than an i3, 3. without the aid of a gpu, 4. without multithreading. 5. without the use of more than one machine.
FINALLY:
To claim your prize, post the original factors of your product here, after the deadline has passed.
And then I'll arrange payment of the prize.
You MUST post the factors of your product after the deadline, to confirm your product and claim your prize.99 -
C/C++ debugging with VSCode sucks ass.
How do I even debug stuff in forks while still debug my threads?
And yeah please just kill my program away, it's not like there happens some necessary housekeeping at exit or SIGINT. FML.8 -
I'm a student at a cyber education program. They taught us Python sockets two weeks ago. The next day, I went home and learned multithreading.
Then, I realized the potential.
I know a guy1 who knows a guy2 who runs a business and could really use an app I could totally make. And it's a great idea and it's gonna be awesome and I'm finally gonna do something useful with my life.
All I gotta do is learn UI. Easy peasy.
I spent the next week or so experimenting with my code, coming up with ideas for the app in my head and of course, telling all my friends about it. Bad habit, I know.
Guy1 was about to meet Guy2, so I asked Guy1 to tell Guy2 about my idea. He agreed. I reminded him again later that day, and then again in a text message.
The next day, I asked him if he remembered.
Guess what.
I asked him to text Guy2 instead. He came back to me with Guy2's reply: "Why won't he send me a message himself?".
So I contacted Guy2. After a while, he replied. We had a short, awkward conversation. Then he asked why he should prefer a new app over the existing replacement.
He activated my trap card. With a long chqin of messages, I unloaded everything I was gathering in my mind for the last week. I explained how he could use the app, what features it could have and how it would solve his problem and improve his product. I finished it off with the good old "Yeah, I was bored😅" to make the whole thing look a bit more casual.
Now, all that's left to do is wait.
...
Out of all the possible outcomes to this situation, this was both the worst the least expected one.
I'm not familliar with the English word for "Two blue checkmarks, no reply". But I'm certain there is no word in any language to describe what I'm feeling about this right now.
By that point, Guy1 has already made it clear that he's not interested in being my messanger anymore. He also told me to let the thing die, just in case I didn't get the hint. I don't blame him though.
It's been almost a week since then. Still no reply from Guy2. I haven't quite been able to get over it. Telling all my friends about it didn't really help.
Looking back, I think Guy2 has never realised he has that problem with his product.
But still, the least he could do is tell me why he dosen't like it...
"Why won't he send me a message himself?" Yeah, why really? HMMM :thinking:
You know what? If I ever somehow get the guts to leave my home country, I'm sending a big "fuck you" to this guy.9 -
oh, I have a few mini-projects I'm proud of. Most of them are just handy utilities easing my BAU Dev/PerfEng/Ops life.
- bthread - multithreading for bash scripts: https://gitlab.com/netikras/bthread
- /dev/rant - a devRant client/device for Linux: https://gitlab.com/netikras/...
- JDBCUtil - a command-line utility to connect to any DB and run arbitrary queries using a JDBC driver: https://gitlab.com/netikras/...
- KubiCon - KuberneterInContainer - does what it says: runs kubernetes inside a container. Makes it super simple to define and extend k8s clusters in simple Dockerfiles: https://gitlab.com/netikras/KubICon
- ws2http - a stateful proxy server simplifying testing websockets - allows you to communicate with websockets using simple HTTP (think: curl, postman or even netcat (nc)): https://gitlab.com/netikras/ws2http -
After a lot of work I figured out how to build the graph component of my LLM. Figured out the basic architecture, how to connect it in, and how to train it. The design and how-to is 100%.
Ironically generating the embeddings is slower than I expect the training itself to take.
A few extensions of the design will also allow bootstrapped and transfer learning, and as a reach, unsupervised learning but I still need to work out the fine details on that.
Right now because of the design of the embeddings (different from standard transformers in a key aspect), they're slow. Like 10 tokens per minute on an i5 (python, no multithreading, no optimization at all, no training on gpu). I've came up with a modification that takes the token embeddings and turns them into hash keys, which should be significantly faster for a variety of reasons. Essentially I generate a tree of all weights, where the parent nodes are the mean of their immediate child nodes, split the tree on lesser-than-greater-than values, and then convert the node values to keys in a hashmap to make lookup very fast.
Weight comparison can be done either directly through tree traversal, or using normalized hamming distance between parent/child weight keys and the lookup weight.
That last bit is designed already and just needs implemented but it is completely doable.
The design itself is 100% attention free incidentally.
I'm outlining the step by step, only the essentials to train a word boundary detector, noun detector, verb detector, as I already considered prior. But now I'm actually able to implement it.
The hard part was figuring out the *graph* part of the model, not the NN part (if you could even call it an NN, which it doesn't fit the definition of, but I don't know what else to call it). Determining what the design would look like, the necessary graph token types, what function they should have, *how* they use the context, how thats calculated, how loss is to be calculated, and how to train it.
I'm happy to report all that is now settled.
I'm hoping to get more work done on it on my day off, but thats seven days away, 9-10 hour shifts, working fucking BurgerKing and all I want to do is program.
And all because no one takes me seriously due to not having a degree.
Fucking aye. What is life.
If I had a laptop and insurance and taxes weren't a thing, I'd go live in my car and code in a fucking mcdonalds or a park all day and not have to give a shit about any of these other externalities like earning minimum wage to pay 25% of it in rent a month and 20% in taxes and other government bullshit.4 -
Cocktail for disaster:
- TDD
- Mocking
- Multithreading
- Averagely well written, testable code
- All tests pass
- One test methods still shows some vague stacktrace in a worker thread ❌ but the test passes ✅
- Run only that test method and no stacktrace.
So I've been pulling my hair for the last two days trying to figure out what was throwing in that test method. Turns out that thanks to the multithreading going on, some other, similar method threw the exception in parallel. And apparently a different test method was already running when the exception was finally caught.
🖕
When I discovered that, it was fixed in a minute. 😭1 -
Today was the first time I used WebWorkers. I loaded it with a hyphenator script because fucking Chrome is still not able to do hyphenation on its own. For the main thread I wrote an injectable Angular service as a wrapper and to enqueue hyphenation work.
So far it works pretty smooth and quickly.
Have you used WebWorkers before? What for?3 -
Daaamn! I needed to process some data simultaneously using PHP, so I thought of using Threads to make things faster, checked out SO and discovered that the available Thread class can only be used in cli environment not on a web server ... FML 😑.
It's like these moments that I remember why I hate PHP, and regret accepting this job.
I miss Java 😣😣6 -
Was writing a multithreaded program in Java with an infinite while loop (for waiting). The thread was never preempted. Added a print statement inside the while loop and everything started working :/
Now I need to print a stupid message just so the program works :P3 -
Today I learned/remember about something I haven't touched in years...
Multithreading and race conditions.
Took me an hour to finally get those locks and numbers to correct/decreasing in order...
Before I got like 73 5 times in a row... -
I need to build multithreading in my brain, i want to learn at least 8 new code languages and techniques at the same time(not to mention normal languages)2
-
My companys custom logging library is not thread safe and has problems with multiple instances of the endproduct as well.1
-
Thank God, most of my clients don't understand multithreading.
Just denied a feature
Reason
1 independent task - 6sec
10 independent tasks - 1min1 -
The company I worked for had to do deletion runs of customer data (files and database records) every year, mainly for legal reasons. Two months before the next run they found out that the next year would bring multiple times the amount of objects, because a decade ago they had introduced a new solution whose data would be eligible for deletion for the first time.
The existing process was not be able to cope with those amounts of objects and froze to death gobbling up every bit of ram on the testing system. So my task was to rewrite the exising code, optimize api calls and somehow I ended up in multithreading the whole process. It worked and is most probably still in production today. 💨 -
!Rant
Didn't have something to do tonight, and I was kind of thinking to make a multicore brainfuckversion for a while, so tonight I implemented it. I call it multiFuck:
https://github.com/papierbouwer/...3 -
i'm afraid that having discovered the power of multithreading has made my code worse.
case in point: me has to calculate an unknown 3rd point of an equilateral triangle many, many times. however, me doesn't get the formula, so me goes ahead and loops over all possible coordinates until it finds the correct one.
yep, it's definitely gotten worse.2 -
for my job I need to know,
Programming, C#, Optimization, Multithreading and Async code, Working certain tools, Reading difficult written code, Understanding, Physics, Networking, Rendering, Codeloops, Memory management, Profiling tools, Being able to make Jira tickets and read Jira tickets. Understanding source control branching, merging, push and pull. bug fixing.
And I write almost 1 line of code a week on average..
I'm a programmer.2 -
Fighting against a occasionally occurring bug, in a networking, multithreading, server-client game.
Thanks to the good excellent front end each client only uses around 20% of my cpu and ~1.3Gb of ram.
Shit happens when you join the development late and your partners code happens on-the-fly.
Maybe redoing it from scratch with something else than JSwing could improve the performanxe slightly -
Just found out that if you spam the different types of notifications that you want to see in my rewrite of devRant you cause an instacrash.
Multithreading is both a blessing and a curse.2 -
So my multithreading optimization is now in UAT.... it looks like it's working and not crashing....
It seems it's also 50%... aka saves 1hr...
I'm having a rare moment when I actually feel good about and actually did something at my job... -
I just realized my male brain is horrible at multithreading. ++ to all womens and their amazing algorithms and capabilities 😘9
-
Programmer looking for a new language
I have been a JavaScript developer for a few years now (non professionally) and I really like the language. I mainly program for execution with NodeJS rather than web, because I feel like I get more freedom (e.i. the ability to use a computer file system).
I normally never use other people's libraries and instead either write my own library/ies for the specific task or use an old one. I only ever use someone else’s if I need a quick frame work to test an idea, never for something I will actually use.
I prefer to work object / class orientated.
I have worked on distributed servers with NodeJS before, however trying to distribute a load across one computer across it's multiple threads has proved problematic due to the heavy delays of standard io transfer speeds.
Why do I want to switch?:
•Because JavaScript is not at all created with multithreading in mind, and pretty much any multithreading solution is a bodge and allot of the time it is more efficient to work single threaded.
•Also, I get the sense that JavaScript + NodeJS is not used often in the programming industry comparison to other languages like; ruby, python, and I don't want to get stuck in a nesh language of which would decrease my employment chances heavy.
Side Note: I have been working on a pet project to have a distributed database (made with nodejs), and so far, there are no language specific problems, but I feel like it would be more efficient if I used a programming language designed more to cater for multi threading.5 -
So, for my task I'm supposed to make Multithreading in java simple and understandable using Runnable classes.
The only problem is, that that's practically impossible when you're in a course where more than 80% can't even write a hello world program.
Any ideas?11 -
My experience was very recent. I was working on my game engine, Pillar3D, and realized that the setup allowed it to be automatically multithreaded with little to no concern about deadlock or race conditions. All based on the assumption that individual levels don't talk to each other, and that moving entities between levels could be done between frames. I can even track about how much work each thread has to do and use that to distribute levels among the threads. Now I can do things like force UI trees to exist in their own level and get fantastic multithreading.
-
Yanno, would be awesome if everything wasn't tied to one main thread. Why is it so much effort to have your ai just run in a separate thread huh4
-
What do you think about the Crystal language?
It looks kinda cool, but I'm somewhat worried about the lack of Multithreading and the beta status so far.21 -
If you guys are ever looking for a chance to shine in the eyes of people you don't respect, may I humbly suggest multithreading a 20 year old project that uses COM and global variables?
I didn't need that hair anyway. -
Is it possible to record the time a thread spends processing only it's code?
E.g. capture sys.ms in thread A -> A is sliced and thread B runs -> B is sliced -> A comes back and captures current sys.ms. The resulting delta of Anow - Ainit includes the time that B spent on the machine.
Is it possible to account for this and get just the time A spent processing?
Is this doable on any other languages?
If it is or isn't, any documentation or papers explain why is appreciated. Google is flooded with "how to time" questions so I'm not seeing any answer for this.7 -
!Rant what's so hard about multithreading for newbie programmers. (An actual question. Trying to tutor some friends who just don't get it)
It's practically a requirement for MVC.
In c# it's extremely simple. Just create a background worker with a handler/interface.2 -
The crazy shenanigans you can do with C++ standard libs are fascinating.
Like implementig multithreading with just a foreach, and bindings which can make member function pointers to simple function pointers, and placeholders in bindings. Also lambda functions are cool.
Something between the lines:
my_crazy_class *tmp = new my_crazy_class(...);
std::vector<type> my_array = .....;
std::for_each(std::execution::par,my_array.begin(),my_array.end(),
[&](type in){
auto fn = std::bind( &my_crazy_class::my_crazy_fnc,*tmp,_1,random_static_value);
return fn(in);
});
ps:
It's pretty much pseudocode, and please don't do things like this, it's bad for your mental health.
pps:
I need to learn how to use this tools wisely. -
Fuck Threads and fuck multithreading in general.
I've spent the last 2 hours and a half looking for an important error that did my project not work at first execution, and it was all because I did start a side Thread, but forgot to wait until it was done before showing the output of it.
Fucking multithreading...3 -
When you realize after an hour of waiting that the script was running on only one of eight cores... but I did get that hour off to read livestream from Apple Event....
-
Found my old project that deals with algebra in multithreading fashion
Just pushed it to github
Have a look at https://github.com/rational-kunal/... -
wish AIs were good at rust, borrowing rules, and async 😫
is it possible to have a impl of async &mut self on something that's gonna thread and update its own data via Arc Mutex or whatever or not
stop making syntax errors
guide pls
nobody uses rust, I swear. or at least they just do basic bitch "beginner" apps. please. get with the times and actually do something meaningful that's not picture perfect theoretical exercises. how come no one's RNG tested every feature against every other feature? where's your chaos monkey. the world is chaos! get with the times!
it would be nice if I stick this on the instance as a method but it _might actually never work_ if I try that so I don't wanna spend 3 days wrangling with the code to figure that out when I have a perfectly good dangling independent helper function in a random package here. gosh darnit
also apparently the only way to get something out of a Arc Murex is to clone it. but the API / usability of the thing would be exactly the same whether it was wrapped in Arc Murex or not. so it's like. if it was in Arc Mutex and you wanna use it in other parts of your app that aren't using multithreading in any way, are you just changing all the function signatures to Arc Mutex or are you cloning to get it back out? uegh I don't even. what if I mutex lock and just put that in the signatures (can I even? because I've tried using weird intermediary objects as part of signatures and then I get in trouble there too cuz arbitrarily the answer is "no" because some generic system limitation)? why all of this
May as well learn hieroglyphics but with French/English grammar exception rules on the side. yo dawg we heard you hate human languages with all their exceptions so we made programming languages the same way70 -
Hello, I have a question for anyone familiar with multithreading!
I just started working with threading for the first time, I mostly write powershell scripts 😅, I found that certain conditions make using multithreading an absolute time saver. And of course in some tasks it's not such a big deal.
I am currently working on a project that runs multiple threads and each thread might invoke one of my functions that also threads the work.
I'm a total newbhat when it comes to this stuff, but if my main process is 4 threads, and I can spin up, up-to 4 more threads to run one of my functions, does the math equate to a possible total number of threads of 16 or is it possible to have the threading go ape-shit bananas and utterly thrash the cpu with rampant threads getting created?
I've looked online and based on some of the info that I've managed to come across on my own, the answers elude towards being safe because I'm creating pools for running the threads first and the pool is responsible for maintaining min/max threads, but I can't seem to find good info on running a pool+threads inside another thread.
Just to let you in on what the function does that requires threading in the first place, I need to basically query CloudTrail based on ARN's to find events, but I can only pass a single ARN to the find-ctevent cmdlet. So I'm essentially making 1500-ish really really small calls to AWS just to get back event data for the ARN.
Serially, this takes like almost 20 mins, on my laptop using stupid settings like 24 threads, it completes in about 95seconds. On the actual server that will be running this code, I'm going to limit it to 4 threads and try to figure out a way to cache the info locally and update the info on a cron or schedule so only the initial scrape takes forever and then the updates can be done nightly or something.
thank you in advance for your help, I'm not too sure if the question is dumb but please let me know either way!8 -
hey guys i need your advice about backend integration for android. I have this junior and I want to teach him about integrating REST API and handling it with volley+caroutines or retrofit+rxjava. currently the junior is using firebase for authentication and firebase realtime db for all CRUD operations. problem is that by using firebase backend the junior won't learn networking/multithreading stuff. Is there a way to use firebase realtime database like a REST API?
-
any advice/suggestions to intensively brush up on modern C++ and multithreading for an interview that will likely be technical and cover bases like algorithms, data structures, etc?
I haven’t done c++ for awhile since a few courses in college - I did parallel programming and GPGPU on the side, but nothing on a professional level.
I’ve been mostly doing front web dev since I got out of school and C#, so I’ve been more on design/higher level of abstraction in dev and if I am asked things about pointers, memory allocations, etc I would probably draw a blank but I am motivated to no life it hard for the next week to catch up again.3 -
!rant
can you give me/point me to some good example problem/exercise for multithreading? as in, something that's small in scope, but actually requires dealing with most of the multithreading issues & complications? race conditions, synchronization, locks, shared memory access, cross-thread calls/callbacks, etc?2