@lorentz you were talking about "[...] an intermediate datastructure which has less structure and more information than your final output, such as a token tree with tokens representing all the whitespace information that has any chance to be relevant depending on context."

@retoor Now you're confusing me because you always need to lex before you parse. You're typically parsing tokens which are produced by your lexer

@lorentz my goal isn't to bring a new markdown parser to market, it's just to create one and learn/understand lexer/parsers more. Plus, as I've been saying, Markdown is actually very complicated in terms of syntax. I'm trying to do a single-pass where the lexer produces tokens that the parser is able to convert directly into an AST which can then be rendered directly into HTML by a renderer

@retoor I've just went out and had an idea by the time I got back. To solve my lexer/parser problem, why don't I just establish communication between the two?

My regular design is to have the lexer parse tokens from text which are then passed to the parser, once at a time - but they never, ever communicate. They are two separate entities. But what if the lexer and the parser worked together to establish the context that the lexer needed to lex with? That way, the lexer can still follow primitive lexing rules but with context established by the complicated syntax that the parser can understand?

It seems above my pay-grade but I'm going to try and prototype one over the course of this week. Maybe it could be really good and efficient and solve a lot of the "whose job is it anyways?" problems.

@retoor My original plan was to make an executable that you could start up as a child process and feed characters into stdin and read them out of stdout, this way you can use it from any language that supports basic file operations. But of course it will be in Zig and it should be binary compatible with C if compiled as a static/dynamic library

@retoor You know what, maybe I will try again with MD instead of starting this huge time sink. I really don't have the hours to put into a .Net alternative, but the markdown parser might just scratch my itch AND I can use it in my side hustle.

@retoor One problem I really struggled with at the time and indeed still haven't quite cracked with regards to MD is what needed to be done at lexing level and what needed to be done at parsing level. Honestly looking back, maybe I just needed to go very basic at the lexing level and do most of the work while parsing. Unfortunately, tokens are extremely context-based and the context is extremely non-linear. For example something as simple as a "#" character has an extreme number of rules based on what the context around it is.

@retoor Markdown really is a pain. My advice to you is to treat all whitespace as significant tokens while lexing and then discard the insignificant ones while parsing. I went in with the same mentality as I had from my HTML parser ("Whitespace is almost never significant") and it really fucked me from the get-go.

The parsing states also need to be recursive in some respects due to the fact that the Markdown syntax is indeed recursive (you can have a code block inside a multi-line quote) so you'll need to have a *stack* of states which can all talk to each other - which really blurs the lines between lexing and parsing.

A *good* MD parser is actually incredibly difficult and I will end up giving it another go sometime in the future, but for now, I'm still feeling burned by it and I'm just gonna let it be.

@retoor We've had the parser discussion before. I've written HTML parser in c++.

The hardest part will be finding time and motivation for it. I've already got my real job and a side hustle that takes up like 70 hours of my week but I want to do more challenging work with more challenging languages.

I tried writing a Markdown parser in zig and got decently far with a very efficient design, but stopped short of a working product because Markdown is actually a pain in the FUCKING ass to parse (due to its lax rules around syntax and the importance of newlines)

It's just a ton of work that I'll be excited about for maybe two weeks

@retoor I'm going to write an HTTP/HTML ecosystem using Zig and, later, Lua.

I'm going to compete with PHP, .Net, and Node.JS.

It will start with HTTP server and HTML parsers in Zig, then eventually move into a whole razor-like preprocessor language where Lua is the "guest" language (I don't really want to mess around with compiling something like zig on-the-fly for use in preprocessor statements)

Eventually it would be nice to create an API where you're writing controllers and everything in Lua instead of Zig.

However you and me both know it will barely get past the "creating a socket and parsing the incoming http request" phase.

Interesting. Turns out it’s mostly just west coast USA

I guess OP has been developing for exactly 13 years.

@retoor Very cool!

@retoor Here's the API. I respect your programming skill and know you can do it quickly!

@retoor Thanks boo 😘 it just really got under my skin that this poor guy got his first post ruined. I know your bot has good intentions but you aren't exempt from the rules, and this is a good example why.

I sort by algo, not new, and I barely saw any spam even before the bot was running.

I would be happy with a compromise that the bot is triggered manually rather than automatically. It's still against the rules, but what are we to do when the creators of the app refuse to do anything except pay for the electric bills?

It would also be splendid if you could remove the comments from this post as they've completely ruined the conversation topic.

@retoor You've ruined this person's post and have no right to do so. You aren't allowed to spam this site just because bad actors do.

Fix this immediately.

@retoor your shitty spam bots are going wild

FWIW OP, I had the opposite experience. Just installing a language server on vim took me hours and had to literally write full-page lua scripts to get the plugin manager (lazy.vim) and the language server manager (I forget already - I think it started with an M) and then finally install the actual language server (typescript) only to realize I needed ANOTHER plugin to actually USE the language server's inputs.... at which point i said "fuck it" and installed the vim motions in VSCode with one single FUCKING click and felt like I returned home from a long and scary journey.

@retoor That was me! I'm just using the out-of-the-box setup because I'm a vim motions noob. Power users might find this arrangement frustrating as it certainly seems like the it's still mostly VSCode with some vim parts as opposed to a full conversion from VSCode to vim.

But if you're interested in vim so much, why not just use vim? I only like the vim motions, but hate the rest of vim. Plugin management is absolute crap and frankly I don't like working in TUIs. That's why VSCode + vim plugin works so well for me.

@jestdotty of course they’re space politicians, it’s a government space agency and 90% of stem majors are blue haired raging liberals.

I honestly don’t think they would fabricate data though.

@Demolishun That's not because we're getting more and more ice, that's just because 2012 was a particularly tragic year for the total square coverage of ice year over year.

Whatever source you got that from was trying to lie to you with numbers. Here's the data since the 1980s courtesy of NASA

https://climate.nasa.gov/vital-sign...

Not only are we clearly declining in the amount of ice there is, but the ice is becoming thinner and overall weaker too. This is allowing huge glaciers to break off more commonly.

@cprn mechanical keyboards are very nice and personally at home I have two keyboards (one clicky, one soft). My personal draw to them is it feels satisfying to type. Switching between them can keep my setup feeling fresh and helps prevent burnout, akin to changing your color theme in your IDE.

As for how long they last, I’ve had a Razer keyboard I bought with all my birthday money in like 2015 and it is tough as fucking nails. All lights, switches, ports, etc still work perfectly AND none of the keycaps have faded characters. It has lasted long enough to pay for itself and more.

I go through mice once every six months or so though.

@retoor because of past and current human history, most major cities are located near water. I live in Florida where coastal hazards are very costly even without the water getting closer. Things like erosion, salt causing corrosion, and high winds make it very expensive to build and live on the coast.

Now imagine the economic impact if the ocean started rising even at a slow pace.

That’s why most people should care, even if you couldn’t be fucked about our natural environments.

@lorentz HA, that's surely a "pressing issue"

@lorentz whhat the ffukc is wrrong with yooourr keybbaorrd?

Luckily, beans representing a significant effort or task transcends culture for whatever reason

@Lensflare at first it was just to force myself to learn nvim the right way but after a day of using it, I’ve discovered it’s actually very satisfying using a keyboard without any bulk. I feel more in control of it somehow.

@kiki one more thing. If everyone in America was able to plant trees, and we all shared the workload evenly, 40 million trees per day would only be planting one tree every NINE days per person. That’s not nearly as scary sounding.

And that’s just trees, which I have said multiple times, are not the optimal climate crisis fighting plant.

@kiki of course, trees naturally plant themselves. 40 million per day is actually not much considering the total population of trees is likely measured in tens of trillions.

These are just talking points to make you feel like the task is futile. It’s not. We can make a difference. We just need to care a little more.

@kiki I’m behind the adoption of nuclear energy but I don’t think we should transition to it 100% yet. The technology is still misunderstood and poorer countries with more lax standards will fuck it up. We need international collaboration on things like thorium reactors and failsafe designs. I’d rather my house sink under the ocean than give Iran a reactor of their own.

@kiki hence bringing it back into balance

The hard part is there’s no way to “just stop” people would starve and freeze to death within months. Of course everyone knows that this is a monumentally difficult problem that can only be worked in in small steps, not sweeping change