You know as an application developer I never saw etl more than mentioned. is that strange ?

Maybe, ETL is almost all I do.
If it ain't ETL it's CRUD.

I much prefer ETL, it has its challenges, but never gets old.

Once you've done CRUD a few thousand times, it's just crud....

@C0D4 lol its just crud :P

is it like a pipeline for data ? sounds like. like something where the entry vectors aren't really the focus just processing a shit ton of data as mass uploaded.

@C0D4 you know maybe this time we could modify this scenario.

need a better job :P you could teach me and we could be bosom companions :P

not like gay stuff

:P

just buddies :)

I think i should get ms strong to blow me again

my direction sense is pulling me where i think i should probably head.

you know since she kept stepping over me for pieces of shit back in the day.

@AvatarOfKaine no homo? 😞

ETL is what it is, take this shit forsaken data from over this way <---- and make it work with this system over that way --->

If you don't like dirty data, then it's probably not for you.

Sometimes, it's bidirectional, that's the stuff that gets your panties all hot and bothered trying to take your own shit forsaken data, and turn it back into that pile of shit the other system gave you in the first place. Somehow every system and process is harder then the last, like every wild developer went "this is my data, and I'll format it it how ever I please"

@C0D4 scratch the no homo part your description has me somewhat bothered. man after my own heart lmao

@C0D4 can someone do something about that girl i have limited history with at the sandwich shop at wegmans and have her put some healthy girth back on and muscle tone ?

i don't know what she did or did not do but its not fair to interpret some stupid color code when I complimented her all those years ago and make her starve herself !!!!

@C0D4 build a framework and reuse your code across projects.

@max19931 🧐the only thing reusable is the load part, usually, at least that's just an api endpoint most of the time.

The extraction and transformation comes in many formats and connecting systems, and when working with MNC's they all have their own shitty way of formatting things or servers and authentications to jump through.

So yea in theory a framework could work, but then that's one wild framework if S3 buckets, ftp and sftp servers , encrypted vs plaintext data (AES, PGP, etc), even that can be per file basis, and god knows what else these days that needs to be feature flagged.

Sometimes it's also a matter of firewalls and setting up dedicated servers to jump out on.

And then going backwards is usually the same thing backwards, at least minus any PII data most of the time.