At the simplest, it takes in a vector of floating-point numbers, multiplies them with other similar vectors (the “weights”), sums each one, applies a RELU* the the result, and then uses those values as a vector for another layer with it’s own weights (or gives output). The magic is in the weights.
This operation is a simple matrix-by-vector product followed by pairwise RELU, if you know what that means.
Where modelWeights is [[[Float]]], and so layer has type [Float] -> [[Float]] -> [Float].
RELU: if i>0 then i else 0. It could also be another nonlinear function, but RELU is obviously fast and works about as well as anything else. There’s interesting theoretical work on certain really weird functions, though.
Less simple, it might have a set pattern of zero weights which can be ignored, allowing fast implementation with a bunch of smaller vectors, or have pairwise multiplication steps, like in the Transformer. Aaand that’s about it, all the rest is stuff that was figured out by trail and error like encoding, and the math behind how to train the weights. Now you know.
Assuming you use hex values for 32-bit weights, you could write a line with 4 no problem:
That’s cool, though honestly I haven’t fully understood, but that’s probably because I don’t know Haskell, that line looked like complete gibberish to me lol. At least I think I got the gist of things on a high level, I’m always curious to understand but never dare to dive deep (holds self from making deep learning joke). Much appriciated btw!
Yeah, maybe somebody can translate for you. I considered using something else, but it was already long and I didn’t feel like writing out multiple loops.
No worries. It’s neat how much such a comparatively simple concept can do, with enough data to work from. Circa-2010 I thought it would never work, lol.
After so many years in this company, lots of the unmaintainable code I have to deal with is either my own fault, or the fault of someone I used to work with but and now they left and I’m the one who has to apologize for their code.
If I move to a different company, 100% of the unmaintainable code I’ll have to deal with there will be someone else’s fault.
And managers don’t like it when you explain that the code is a unmanageable mess because they put a deadline on every goddamn thing and never pay off technical debt.
At a new place you can honestly say “the code is kinda a mess, it needs a bunch of work” and the manager can just assume it was because the last guy didn’t know what he was doing and not because of their own shitty management.
Management could implement a code review process to avoid this.
Software development isn’t a brand new field anymore. Most problems are well known and therefore have well known solutions. So it pretty much always comes down to management not wanting to implement the known solutions to the problems because its easier to blame the devs.
Perl also has unless() for the very purpose in OP, which is a more sensible choice.
Oh, and if you need to reinforce your belief that Perl is a mess, the single-quote character can be used as a package separator instead of “::”. This was set in the 90s when nobody was quite sure of the right syntax for package separators, so it borrowed “::” from C++ and the single quote from Ada (I think).
That means the ifn’t() in OP can be interpreted as calling the t() function on the ifn package.
The “::” separator is vastly preferred, though. Single quotes run havoc on syntax highlighting text editors (since they can also be used for strings). About the only time I’ve seen it used is a joke module, Acme::don’t.
Learning to deal with “unmaintanable” codebases is a pretty good skill. It taught me good documentation and refactoring manners. It’s only a problem for you if management does not accept that their velocity has gone down as a result of tech debt pilling up.
Code should scream it’s intent (business-wise) so as to be self-documenting as much as possible As much as possible is not 100%, so add comments when needed. Comments should be assumed to be relevant when written, at best. Git comment should be linked to your work ticket so that we can figure out why the hell you would do that, when looking at the code file itself. I swear some people seem to think we only read them in PRs (we don’t). Overall concepts used everyday, if they need to be reexplained, should probably be written down (at least today’s version). Tests are documentation. Often the only up to date one?
Recently switched jobs from maintaining a 15 year old Windows Forms .NET Framework legacy codebase.
At the new job we stick to Clean Architecture, use unit and integration tests, have a code generation tool, actually make nice use of generics and use dependency injection. Also agile processes, automatic build tools, whatever. The difference is night and day and I’m so glad my ex boss fired me because I told him he’s an asshole and his codebase is shit.
My first job out of college I have been able to see a steady improvement in the codebase. A little while ago I had to go back to an old tag and was horrified with what it used to be and impressed how much it improved.
This reminds me of something I saw online maybe 20 years ago now. Someone created a torrent with a name like “every IP address ever (hacking tool)” and uploaded it to Suprnova, which ended up having thousands of people seeding it. It was just a text file with every IPv4 from 0.0.0.0 to 255.255.255.255 😂
programming.dev
Active