Heres some research into a new LLM architecture I recently built and have had actual success with.

The idea is simple, you do the standard thing of generating random vectors for your dictionary of tokens, we'll call these numbers your 'weights'. Then, for whatever sentence you want to use as input, you generate a context embedding by looking up those tokens, and putting them into a list.

Next, you do the same for the output you want to map to, lets call it the decoder embedding.

You then loop, and generate a 'noise embedding', for each vector or individual token in the context embedding, you then subtract that token's noise value from that token's embedding value or specific weight.

You find the weight index in the weight dictionary (one entry per word or token in your token dictionary) thats closest to this embedding. You use a version of cuckoo hashing where similar values are stored near each other, and the canonical weight values are actually the key of each key:value pair in your token dictionary. When doing this you align all random numbered keys in the dictionary (a uniform sample from 0 to 1), and look at hamming distance between the context embedding+noise embedding (called the encoder embedding) versus the canonical keys, with each digit from left to right being penalized by some factor f (because numbers further left are larger magnitudes), and then penalize or reward based on the numeric closeness of any given individual digit of the encoder embedding at the same index of any given weight i.

You then substitute the canonical weight in place of this encoder embedding, look up that weights index in my earliest version, and then use that index to lookup the word|token in the token dictionary and compare it to the word at the current index of the training output to match against.

Of course by switching to the hash version the lookup is significantly faster, but I digress.

That introduces a problem.
If each input token matches one output token how do we get variable length outputs, how do we do n-to-m mappings of input and output?

One of the things I explored was using pseudo-markovian processes, where theres one node, A, with two links to itself, B, and C.
B is a transition matrix, and A holds its own state. At any given timestep, A may use either the default transition matrix (training data encoder embeddings) with B, or it may generate new ones, using C and a context window of A's prior states.

C can be used to modify A, or it can be used to as a noise embedding to modify B.

A can take on the state of both A and C or A and B. In fact we do both, and measure which is closest to the correct output during training.

What this *doesn't* do is give us variable length encodings or decodings.

So I thought a while and said, if we're using noise embeddings, why can't we use multiple?

And if we're doing multiple, what if we used a middle layer, lets call it the 'key', and took its mean
over *many* training examples, and used it to map from the variance of an input (query) to the variance and mean of
a training or inference output (value).

But how does that tell us when to stop or continue generating tokens for the output?

Posted on pastebin if you want to read the whole thing (DR wouldn't post for some reason).

In any case I wasn't sure if I was dreaming or if I was off in left field, so I went and built the damn thing, the autoencoder part, wasn't even sure I could, but I did, and it just works. I'm still scratching my head.

https://pastebin.com/xAHRhmfH

Kiki’s meme of the day

GSUS XRIST! from 3.7M to 96K o.O that's Ocaml's compiler for you 🤤

What are the key differences between a large language model and traditional machine learning models in terms of architecture and application?

Follow-up: How do these differences impact the model's ability to understand and generate human-like text?

team lead/senior telling you that you're probably going to break prod and have to patch it for the next couple of weeks when working on one of the first migration epics on legacy monolith, but we'll get good data from it doesn't make it any less terrifying

Checkout this random bullshit code I found:

Bible verses.

Sorted from "wtf" to progressively more disturbing.

---
1️⃣ One of many contradictions
---

John 4:12
"No one has seen the face of God."

Genesis 32:31
"I have seen the face of God..."

---
2️⃣ All-Knowing God, regrets creating you
---

Genesis 6:6
"And it repented the LORD that he had made man on the earth, and it grieved him at his heart."

---
3️⃣ Says to kill all gay people
---

Leviticus 20:13
"If a man lies with a male as with a woman, they have committed an abomination; the two of them shall be put to death; their bloodguilt is upon them."

---
4️⃣ Approval of sex trafficking
---

Exodus 21:7
"If a man sells his daughter as a servant, she is not to go free as male servants do. If she does not please the master who has selected her for himself, he must let her be redeemed."

---
5️⃣ Says its ok to own slaves
---

Exodus 21:20-21
"Anyone who beats their male or female slave with a rod must be punished if the slave dies as a direct result, but they are not to be punished if the slave recovers after a day or two, since the slave is their property."

---
6️⃣ Says its ok to kill children
---

Psalms 137:9
"Blessed the one who seizes your children and smashes them against the rock: the children represent the future generations, and so must be destroyed if the enemy is truly to be eradicated."

---
7️⃣ Says a raped woman must marry her rapist
---

Deuteronomy 22:28–29
"If a man meets a virgin who is not betrothed, and seizes her and lies with her, and they are found, then the man who lay with her shall give to the father of the young woman fifty shekels of silver, and she shall be his wife, because he has violated her. He may not divorce her all his days."

---
8️⃣ Says incest is ok
---

Genesis 19:32-36
"Let’s get our father to drink wine and then sleep with him and preserve our family line through our father. That night they got their father to drink wine, and the older daughter went in and slept with him. He was not aware of it when she lay down or when she got up. The next day the older daughter said to the younger, “Last night I slept with my father. Let’s get him to drink wine again tonight, and you go in and sleep with him so we can preserve our family line through our father.” So they got their father to drink wine that night also, and the younger daughter went in and slept with him. Again he was not aware of it when she lay down or when she got up. So both of Lot’s daughters became pregnant by their father."

v0.0005a (alpha)
- class support added to lua thanks to yonaba.
- rkUIs class created
- new panel class
- added drawing code for panel
- fixed bug where some sides of the UI's border were failing to drawing (line rendering quark)

v0.0014a (alpha) 11.30.2023 (~2 hours)
- successfully retrieving basic data from save folder, load text into lua from files
- added 'props' property to Entity class
- added a props table to control what gets serialized and what doesn't
- added a save() base method for instances (has to be overridden to be useful beyond the basics)
- moved the lume.serialize() call into the :save() method on the base entity class itself
- serialized and successfully saved an entities property table.
- fixed deserializion bugs involving wrong indexes (savedata[1] not savedata[2])
- moved deserialization from temp code, into line loading loop itself (assuming each item is on one line)
- deser'd test data, and init()'d new player Entity using the freshly-loaded data, and displayed the entity sprite

All in all not a bad session. Understanding filing handling and how to interact with the directory system was the biggest hurdle I was worried about for building my tools.

Next steps will be defining some basic UI elements (with overridable draw code), and then loading and initializing the UI from lua or json.

New projects can be set as subfolders folders in appdata, using 'Setidentity("appname/projectname") to keep things clean.

I'm not even dreading writing basic syntax highlighting!

Idea is to dogfood the whole process. UI is in-engine rendered just like you might see with godot, unity, or gamemaker, that way I have maximum flexibility to style it the way I want. I'm familiar enough with constructing from polygons, on top of stenciling, on top of nine-slicing, on top of existing tweening and special effects, that I can achieve exactly what I want.

Idea is to build a really well managed asset pipeline. Stencyl, as 'crappy' as it appeared, and 'for education' was a master class in how to do things the correct way, it was just horribly bloated while doing it.

Logical tilesets that you import, can rearrange through drag-n-drop, assign custom tile shapes to, physics materials, collisions groups, name, add tag data to, all in one editor? Yes please.
Every other 2D editor is basic-bitch, has you importing images, and at most generates different scales and does the slicing for you.

Code editor? Everything behavior was in a component, with custom fields. All your code goes into a list of events, which you can toggle on and off with a proper toggle button, so you can explicitly experiment, instead of commenting shit out (yes git is better, but we're talking solo amateurs here, they're not gonna be using git out the gate unless they already know what they're doing).

Components all have an image assignable to identify them, along with a description field, and they're arranged in a 2d grid for easy browsing, copying, modifying.

The physics shape editor, the animation editor, the map editor, all of it was so bare bones and yet had things others didn't.

I want that, except without the historic ties to flash, without the overhead of java, and with sexier fucking in-engine rendering of the UI and support for modding and in-engine custom tools.

Not really doing it for anyone except myself, and doubt I'll get very far, but since I dropped looking for easy solutions, I've just been powering through all the areas I don't understand and doing the work.

I rediscovered my love of programming after 3-4 years of learning to hate it, and things are looking up.

Turns out composition over inheritance won't save you from downcast hell, it just becomes `_ => abort()` hell.

Hi FuLlStAcKcLoWn,

I fOuNd A fEw ErRoRs On YoUr SiTe.

MaY i SeNd OvEr SoMe Of ThOsE eRrOrS aNd AuDiT rEpOrT wItH pRiCeS?

i’m LoOkInG fOrWaRd To YoUr RePlY

bEsT rEgArDs

god this is getting repetitive, how can we spam the spammers? wasn't there a site like "send your enemies glitter"? classic!

i guess this is what I get for putting my company email public

but really, what happened to gmail's spam filter? should spot ones like these from miles away

It is with great pleasure that I announce I have just been appointed the CEO of Garalina.

With all those amazing people we have on board, we as a company will carry on delivering what we do best — the entertainment meets therapy. With your support, we can evolve and extend our reach way beyond the select few families we had a pleasure to work with.

Stay tuned!

Design in Motion: Real-Time Rendering's Impact on Architecture

Architecture, a discipline that once relied heavily on blueprints, models, and lengthy render times, has undergone a revolutionary transformation in recent years. The advent of real-time rendering technology has fundamentally altered the way architects visualize, present, and interact with their designs. This paradigm shift has not only enhanced the creative process but has also empowered architects to make more informed decisions and create immersive experiences for clients and stakeholders.

Real-time rendering, a technological marvel that harnesses the power of high-performance graphics hardware and advanced software algorithms, allows architects to generate photorealistic visualizations of their designs in a matter of milliseconds. Gone are the days of waiting hours or even days for a single rendering to complete. This acceleration in rendering time has not only expedited the design process but has also encouraged architects to explore multiple design iterations rapidly.

One of the most significant impacts of real-time rendering on architecture is the ability to visualize a design in various lighting conditions and environmental settings. Architects can now instantly switch between daytime and nighttime lighting scenarios, experiment with different materials, and observe how their designs respond to different seasons or weather conditions. This level of dynamic visualization offers insights into how a building's appearance and functionality evolve throughout the day, contributing to more holistic and thoughtful design solutions.

Moreover, real-time rendering has transformed client presentations. Architectural concepts can now be communicated with unprecedented clarity and realism. Clients can virtually walk through spaces, observing intricate details, exploring different angles, and even experiencing the play of light and shadow in real-time. This immersive experience fosters a deeper understanding of the design intent, enabling clients to provide more targeted feedback and make informed decisions.

The impact of real-time rendering on collaboration within architectural teams cannot be overstated. Traditionally, architects and designers would need to wait for a rendering to complete before discussing design changes or improvements. With real-time rendering, team members can make adjustments on the fly, observing the immediate effects of their decisions. This seamless collaboration not only enhances efficiency but also encourages interdisciplinary collaboration as architects, engineers, and other stakeholders can work together in real-time to refine designs.

The integration of virtual reality (VR) and augmented reality (AR) into the architectural workflow is another transformative aspect of real-time rendering. Architects can now create VR environments that allow clients to step inside their designs and explore every nook and cranny. This not only enhances client engagement but also enables architects to identify potential design flaws or spatial issues that might not be apparent in 2D drawings. AR, on the other hand, overlays digital information onto the physical world, facilitating on-site decision-making and construction supervision.

Real-time rendering's impact extends beyond the design phase. It has proven to be a valuable tool for public engagement and community involvement in architectural projects. By creating virtual walkthroughs of proposed structures, architects can offer the public an opportunity to experience the design before construction begins. This transparency fosters a sense of ownership and allows for constructive feedback, contributing to the development of designs that resonate with the community's needs and aspirations.

The environmental implications of real-time rendering are also noteworthy. The ability to visualize designs in various environmental contexts contributes to more sustainable architecture. Architects can assess how natural light interacts with interior spaces, optimizing energy efficiency and reducing the need for artificial lighting during the day.

In conclusion, real-time rendering has ushered in a new era of architectural design, propelling the industry into a realm of dynamic visualization, immersive experiences, and enhanced collaboration. The ability to witness designs in motion, explore different lighting conditions, and interact with virtual environments has redefined how architects approach their craft. From facilitating client presentations to fostering sustainable design solutions, real-time rendering's impact on architecture is profound and multifaceted. As the technology continues to evolve, architects have an unprecedented opportunity to push the boundaries of creativity, efficiency, and sustainability in the built environment.

I'm a Newbie to networking, and currently trying to understand this Network and Identifying the IPs for which interfaces.

I'd love to know which IP address is for the captured Router interface, etc. Which IP is the address of the captured interface n2.

How do I approach to solve this Problem?

I used to design and program these systems. Some of the first voice responsive ones. I’m so sorry for my crimes against humanity.

an amazing video about intel euclid by an amazing creator. She also has vids where she hacks a DVD drive into a laser engraver!
https://diode.zone/w/...

Swift 5.9 will get huge additions to the language, compared to the current version 5.8, including type safe and checked macros much like in Rust.
And it’s still not going to be a major version bump to 6.0!
I’m thrilled to see what kind of stuff will be in 6.0 then! 😄

This is cool as hell:
http://www.quiss.org/boo/

That is all.

does recursion have any practical use outside of being a cute/elegant solution under constraints where stack overflow isn't a concern due to small input size, and leetcode?

im having trouble thinking of anywhere you could justify using recursion in industry outside of leetcoding people

i assume the iterative approach would be preferred in scenarios where scaling matters

Windows 10 forces Chrome & other browsers to perform slow and laggy so that dumbass people can start switching to their fucking Edge.

How fucked up is fucked up - Part 1.

What's your most frustrating moment for developing software from scratch in a start up company?

Pro tip: never try to convert the date (formatted as "yyMMddHHmm") in an int32.
It will stop working in 2022. 😉

I just found out I accidentally invented the "2022 bug" two years ago.

Satoshi just pushed another commit on:
https://github.com/bitcoin/bitcoin/...

Anyone know of any companies/agencies that are looking for freelance technical writers, for blog posts and such?

Mainly within the areas of DevOps, CI/CD, Kubernetes, bash etc.

Guys...watch the cppcon21 keynotes.
That is good stuff.

The insights and ideas in these talks are not just for c++. They apply to CS in general.

The presenters have done a phenomenal job with the content. There is a good deal of philosophy for what it means to code and how it should look like (not specifically in c++). Most of it is distilled insights collated right from the times when computer science was a domain of mathematicians and ee majors to the modern age.
It is like Dr.Dobbs but in video. Even book like. Certainly very dense.

A puzzle, just for fun.

Two friends, (a)lice and (b)ob are communicating through a channel encrypted with random numbers XOR'd together, like so:

keyA = randint(1024, 1024**2)
keyB = randint(1024, 1024**2)

msg = randint(1024, 1024**2)

You, an interloper, have watched all these communications, siphoning the packets as they went.

When alice sends a message to bob's mailbox, she does it like so:

mailBoxB = keyA^msg

Bob's mailbox receives the mail automatically, and applies his own key, sending it back to alice's mailbox:

mailBoxA = keyB^mailBoxB

Next, Alice's mailbox notices the message, and automatically removes her key and sends it back to bob's mailbox. All of this, the first message, the second, and the third, happens in milliseconds, the back and forth.

mailBoxB2 = mailBoxA^keyA

Finally, bob's mailbox removes his key, and deposits the now unencrypted message in his box, for him to read in the morning:

mailBoxBFinal = mailBoxB2^keyB

As as a spy, you know the first packet sent to bob, had a value of 589505.

The packet bob sent back to alice, after applying his key, has a value of 326166

The message sent *back* to bob after alice removed *her* key, had a value of:
576941

What are the values of keyA, keyB, and what is the value of the msg?

New useless information unlocked

Is this normal?

"not boolean" statement can be written as "(boolean+(-1))**2". Pls comment a better alternative lmao

soooo Texas froze before Hell, thought it was never going to happen but it did 🥴

Had a meeting scheduled for today, but since my employees (crucial for the meeting) do not have power I had to reschedule.

VP: I saw that you cancelled the meeting
Me: yes I did sir, my people do not have power at their homes so I decided that we can postpone it for later.

VP: Understandable, I just wanted to see if you guys were ok.

Me: yes sir, we are thank you and yourself?

The rest of the conv was standard pleasantries.

Why can't it be like this for all devs around the world?

Unicode I love you!

Who the hell invented the unicode character 'ZERO WIDTH SPACE'

https://fileformat.info/info/...