making your own tools to solve your own problems is the tech version of teaching yourself how to fish

i have never agreed with the sentiment “don’t spend an hour automating a 5 minute problem”

it always felt like context-unaware fearmongering

the main reason is that time isn’t the only factor, if time was literally the only thing at play then nobody would make CI/CD pipelines because way faster and easier to just build your docker image locally with your beefy CPU and manually click-drag it into the gui for your images

Time

time isn’t a good metric here, because time isn’t the only factor, we aren’t procedure-driven soulless robots, so those 5 minutes that sound so trivial actually weigh on you in different ways, context switch you, are tedious, and make you feel like crap overall

that is poison to your productivity no matter how you look at it

if you spend an hour spent automating a thing, you’ll be:

• refining your programming skills
• dealing with input
• doing api calls
• parsing files
• reacting to weird edge cases

you’ll be collecting exp the whole time, and at the end of it you never have to deal with the burden of that five minute task again, win-win.

try to automate everything, pre-solve every one of your problems, eliminate actual waste of time “busy work” by spending a little time just fixing the problem

as an added bonus, you never need to solve that problem again, just pull in the function that does the thing and you’re off to the races

What to automate?

how many platforms have an api that you can hit and do ten things in the same time as it takes for someone to do one in a gui

how many crazy “multi-month” horror tasks can be totally eliminated with a pile of sufficiently gross one-off regex

at my last job there was a horror task like this, nobody wanted to tackle it because it seemed like it would be manual trawling through close to internal 1000 repos, categorizing them based on a long list of data points, and making a call about each

should it be archived? is it even deployed anymore? is it using up-to-date pipelines? are the dependencies updated? does it have a codeowner who still works for the company?

while everybody else was freaking out at the prospect of this monolithic task, and figuring out ways to deploy something into ci which could scan for some of these things and send it to an internal analytics platform and notify us in alerts when something was wrong

i just wrote a script

i grabbed a recent backup of every repo, unzipped it, and wrote a tool that walked every single repo, scanned every one of the required data points, and produced a report on each, then dumped it all into a giant csv for anyone who wanted to see

know what we found out? we were asking the wrong questions, half the datapoints weren’t useful, it wouldn’t have worked in the originally planned way

my script that i cooked up over lunch and a coffee not only eliminated many weeks of building a crazy deployed pipeline system, but also potentially many weeks of yapping about all the things that system even needed to be

nobody asked me to, it just seemed like an easy problem to brute force

the script took 13 minutes to run, wasn’t optimized, relied on every repo to be in the filesystem, was a pile of gross regex and chaotic specific solutions.

but it worked

Tooling up your tooling

your solution doesn’t need to be reusable or “production ready”, we’re not thinking about deployment here — this thing can be an absolute mess, inefficient, memory leaks all over the place, who cares you are just running it one time

do it recursively. automate automation, tool up your tooling, if it isn’t an interesting problem then make an interesting problem out of it and walk away with it never being your problem again

when you get good at writing quick and dirty scripts, you find yourself writing the same chunks over and over again:

• read a config file (json/yaml/toml)
• parse cli arguments
• ask the user questions and store responses
• make calls to a remote api
• scan a directory
• write output to a file (csv/yaml/whatever)
• display something in a digestible format

so why don’t we write all of those things, do it once and then just use it forever, your own personal “trash script” library, minimize friction in writing new scripts

i have one of these and it massively reduces the time for me to build a new automation, it is such a wasteland of old code snippets and occasionally useful functions, i can build just about any terminal system that i want with it

it has all of my keys, can talk to all of the apis that i care about

need it to be a server? easy.
need it to be a globally callable cli so i can use it in a directory of my choosing? done.
need it to ask me a bunch of questions? solved.
need it to parse some positional/named args and fill in the blanks? got it covered.

build yourself a wild library of all the problems you have ever solved and it isn’t “an hour to solve a 5 minute problem” it starts looking like “2 minutes to solve a 5 minute problem”

Levelling up

if you haven’t ever built your own tooling it can feel intimidating, always relying on libraries authored by genius “cracked” engineers who put in countless hours to solve every single edge-case for this one thing, it seems impossible

but that isn’t what we’re trying to do

none of this is magic, all of programming is just variables and loops and inputs and outputs and interfaces - there is no magic, nothing that exists so far outside of what is possible that you couldn’t just sit down and figure it out

practice by taking a thing that seems like magic and just doing it, break the problem down, start at the most basic concept and slowly carve out more and more of the problem domain

it’s easier than you think

the best exp gains come from the gotchas not the happy path, edge cases and weird implementation details are where we grow the most as developers, if something seems like magic then dive into what it is actually doing, figure things out from the ground up

most things are just compositions of simple components

About solutions

when you use someone else’s tool, sometimes that thing will be so heavy and complex, use all of your system resources, and only really be doing the simplest of operations, why is that?

everything is trying to be one size fits all.

everything available on the open market is meant to satisfy every possible use case, to scale infinitely, to work on every machine and every operating system, to make all of your problems magically disappear, to fit every niche concern of every user who has ever wanted to run it

you don’t need that, your solution only needs to fit your problem

go bash out a gross shell/node/py/whatever script, who cares. solve the problem you are having, exactly that problem, don’t productionize it in the hopes that your thing can fit to every other human alive

this isn’t a product, this is your tool

a single tool for a single job, doesn’t need to be customizable, doesn’t need all the bells and whistles, doesn’t need error checking or logging or metrics or validation or testing

your tool should solve your problem, nothing more, nothing less

Solving problems

i have a crazy toolbox for mostly cli problem solving (details below), but also sometimes i just want to solve a specific problem because everything out there feels bad

maybe other solutions are trying too hard, or doing too much, or they don’t fit my niche use case, whatever the case may be developing a muscle for just rolling your own solution can be incredibly powerful

i recently did this with the problem i was having getting exact images at specific aspect ratios, it is tough to deal with aspect ratios because you’re juggling the dimensions of your image and trying to fit that mentally into one or both of the numbers of the ratio, and then like deciding how to crop the image etc.

so i made a tool, eliminated the mental overhead, turned a slightly frustrating problem into a nice user interface, learned some things along the way

https://aspect.alia.tools // aspect ratio photo editor

it took a few hours, but in that few hours i not only solved my problem but i came out of it with a nice interface for doing it repeatably in no time at all

now that problem is gone, and i have a cool image editor, other people can use this image editor if they like

sometimes a solution just doesn’t exist

if you are working on something really niche and esoteric, a tool is not only a game-changer, but might be the only option

i was building an OSC-based lipsync system for live performers using digital avatars, that just didn’t exist

what i had to build was completely custom and half-way between a domain-specific language for handling mouth movements over timed intervals, and a music mapping tool like what would be used for user-generated custom osu/ddr/beat saber levels

such an interesting problem, this tool is the only thing that solves it, had a blast making it, took a couple days, learned a lot, 100% worth

My toolbox

speaking more personally, my toolbox is a folder full of scripts, i have an entry point which will match the first cli argument to the name of one of these scripts, and pass along any other arguments into that script

the scripts themselves run in a wrapper which provides a lot of the boilerplate functionality, and they call on a giant library of snippets/interfaces/common operations

q echo "hello world"

my toolbox is called q, not for any reason, it is just close to the home row, a single letter, isn’t anything by default, saves some time. “echo” would be the name of the script, and then echo will receive “hello world” as an argument

if i just typed q echo then it will ask me what i’d like to echo, this is baked into the wrapper, arguments can be positional, they can be named, if nothing is specified then the system will prompt to fill in the details

this script is super basic, it will just output whatever the input is, who even needs this? but it is a good “hello world” example

q llm qwen2 "what is a mango?"

this one is slightly more complicated under the hood, but does all the same things, if i miss out which llm i want to use it will prompt me for ones i have configured, if i miss out the query it will ask me, all the script does is talks to the llm’s endpoint and returns the answer, but that is really useful as a tool

speaking to an llm is part of my library, argument parsing is part of the wrapper

two basic examples
but the point is that they don’t take any time to write because i already wrote all of the library code, trying to find something “off the shelf” might turn into hours of messing around with configuration and dependencies and updating things, too long

for me, the framework is already there, my system has a way for me to add new one-off scripts, those scripts have access to each other and every piece of library code i’ve ever written, every api integration, every external dependency

this is my playground, my sandbox, my toolkit

it isn’t made for anyone else to use, it isn’t sane enough for anyone else to even learn, but it helps me move 10+ times faster, it gives me a wealth of snippets and functionality that i can just get running with

before most people finish initializing a new project to solve a problem i’ve already got a working solution

be more unhinged, make a giant inhuman project for yourself, make a wild giant gui with 100 random tools baked in, solve a bunch of problems, stop over-thinking, stop adding complexity, stop getting in your own way

make your own tools

〆 alia