Do all the things like ++ or -- rants, post your own rants, comment on others' rants and build your customized dev avatarSign Up
From the creators of devRant, Pipeless lets you power real-time personalized recommendations and activity feeds using a simple APILearn More
Search - "traefik"
If you've ever tried using Go plugins raise your hand.
If you've ever tried doing plugins in Go, raise your hand.
If you think that the following rant will be interesting, raise your hand.
If you raised your hand, press [Read More]:
This is a tale of pain and sorrow, the sorrow of discovering that what could be a wonderful feature is woefully incomplete, and won't be for a very long time...
Go plugins are a cool feature: dynamically load pre-compiled code, and interact with it in a useful and relatively performant way (e.g. for dynamically extending the capabilities of your program). So far it sounds great, I know right?
Now let me list off some issues (in order of me remembering them):
1. You can't unload them (due to some bs about dlopen), so you need to restart the application...
2. They bundle the stdlib like a regular Go binary, despite the fact that they're meant to be dynamic!
3. #2 wouldn't be so bad if they didn't also require identical versions of all dependencies in both binaries (meaning you'd need to vendor the dependencies, and also hope you are using the right Go version).
4. You need to use -trimpath or everything dies...
All in all, they are broken and no one is rushing to fix it (literally, the Go team said they aren't really supporting it currently...).
So what other options are there for making plugins in Go?
There's the Hashicorp method of using RPC, where you have two separate applications one the plugin, one the plugin server, and they communicate over RPC. I don't like it. Why? Because it feels like a hack, it's not really efficient and it carries a fear of a limitation that I don't like...
Then we come to a somewhat more clever approach: using Lua (or any other scripting language), it's well known, it's what everyone uses (at least in games...). But, it simply is too hard to use, all the Go Lua VMs I could find were simply too hard to set up...
Now we come to the most creative option I've seen yet: WASM. Now you ask "WASM!? But that's a web thing, how are you gonna make that work?" Indeed, my son, it is a web thing, but that doesn't mean I can't use it! Someone made a WASM VM for Go, and the pros are that you can use any WASM supporting language (i.e. any/all of them). Problem inefficient, PITA to use, and also suffers from the same issues that were preventing me from using Lua.
Enter Yaegi, a Go interpreter created by the same guys who made (and named) Traefik. Yes, you heard me right, an INTERPRETER (i.e. like python) so while it's not super performant (and possibly suffering from large inefficiency issues), it's very easy to set up, and it means that my plugins can still be written in Go (yay)! However, don't think this method doesn't have its own issues, there's still the problem of effectively abstracting different types of plugins without requiring too much boilerplate (a hard problem that I'm actively working on, commits coming soon). However, this still feels to be the best option.
As you can see, doing plugins in Go is a very hard problem. In the coming weeks (hopefully), I'm going to (attempt to at least) benchmark all the different options, as well as publish a library that should help make using Yaegi based plugins easier. All of this stuff will go (see what I did there 😉) in a nice blog post that better explains the issues and solutions. But until then I have some coding to do...
Have a good night(/day)!13
"DO NOT USE ISSUES FOR GENERAL SUPPORT.
Issues are for request only. You can use the following:
- Stack Overflow (using the "traefik" tag) (where no question excepted the simplest are ever answered)
- the Traefik community Slack channel (where no one is online)
So, what should I put as a title for my "issue"...1
Just found a tutorial on DigitalOcean to setup traefik so I can easily make my images accessible to the outside.
"So create this file, add this in it, create this proxy, run this long ass command, and when you go to 'https://monitor.example.com', you should get this dashboard"
Got "This website is not available".
I've ranted about this before, but here we go again:
I was racking my brains trying to figure out how one could possibly implement plugins easily in Go.
I had a look at using RPC, which requires far to much boilerplate to be realistic. I looked at using Lua, but there doesn't seem to be a straight forward way of using it. I was even about to go with using WASM (yes, WASM). But then I came across Yaegi ("Yet another elegant Go interpreter", you heard right: "interpreter"), Yaegi is also very easy to use.
There are a few issues (including some I haven't solved yet), including flexibility (multiple types of plugins), module support, etc. Fortunately, Traefik just released their plugin system which is based on Yaegi (same company), and I got to learn a few tricks from them.
Here's how module loading works: The developer vendors their dependencies and pushes them to a repo. The user downloads the repo as a zip and saves it to the plugins folder. I hash the zip, unzip it to a cache, and set the the GOPATH for the interpreter to be that extracted folder. I then load the module (which is defined by a config file in the folder), and save it for later. This is the relatively easy part.
The hard part is allowing for different types of plugins. It looks easy, but Go has a strict typing system, makes things complicated. I'm in the process of solving this problem, and so far it should go like this: Check that the plugin fits an arbitrary interface, and if it does, we're good the go. I will just have to apply the returned plugin to that interface. I don't like this method for a few reasons, but hopefully with generics it will become a bit more clean.1
For me, it would be Caddy Server
Discovered it while running some researches around GoLang, where I could see that it was really useful when used with Hugo Framework.
It took me something like 10mn to configure it to run (I couldn't believe only 2 lines would make my things work), and 5 more to make it run with Docker and Traefik (another good open source project that I now use everyday).
Now all my projects where I use Docker have Caddy included o/
While planning my (personal) server I just seem to pile up more and more things to do/consider. Basically, for now I just want to have rclone, nextcloud and jellyfin, plus some usenet stuff later on. But I want to have the whole installation and configuration automated as far as possible, since I'll at first it will run in a test environment and needs to be migrated to another server at a point, possibly even another OS. So I suppose that means docker, docker-compose and Chef (any better options?). I want SSL: Traefik. User management / auth? RADIUS, LDAP. SSO? keycloak. I also need to deal with virtual hosts. And probably much more..
Since I just have basic Linux knowledge and have no real experience with any of the other technologies, I feel a bit lost. I just got to the abovementioned software due to some ddg research. I don't mind digging deep, I want to learn (which is half the reason for this project), but it's not easy to the the best way to set this up.11
So remember rant from yesterday about Go plugins? Well turns out that a new version of Traefik was just released with plugins based on Yaegi!
"It can't all be good news" you say, suddenly paying attention. You're right, their implementation is a bit weird and I don't fully understand it... So I've decided to try to ask for help from the guy who wrote the feature, let's see if he responds...
If anyone is looking for a great tutorial on getting started with a docker cluster check out https://dockerswarm.rocks/
I had a 4 node cluster up on Digital Ocean with Traefik + Lets Encrypt, Prometheus, Portainer, Grafana all that good stuff in under 2 hours. Not much longer to test a basic WP and Next Cloud container with full SSL. Neat stuff. Just burning through $100 credit for testing but it's been fun5
I've got a somewhat special issue with my setup.
I am running an instance of `lucaslorentz/ caddy-docker-proxy` as proxy that handles certificates and request and proxies them to docker containers that run `abiosoft/caddy:php` to host Laravel based applications. The problem is, that the `abiosoft/caddy` containers do not know it's assigned hostname and thus Laravel's `asset`, `secure_asset` and `url` respectively `secure_url` don't work as they use the internal hostname which would be an IP address and thus requests go to 192.168.240.x instead of example.com.
I am not yet entirely sure where I should tackle this problem and am grateful for every hint.
I am currently also evaluating traefik instead of Caddy-docker-Proxy and Caddys v2 official container instead of abiosoft's Caddy v1 container but I guess, that this wouldn't solve the issue as the container still wouldn't know that it's given Domainname is example.com4
Can traefik's documentation and configuration be any more confusing and retarded? I hate this thing. Any good alternatives?1