This post was not created with the help of an LLM. Thanks for stopping by.
I am no stranger to using LLMs to aid in my development process. This is not an anti-LLM post. It's a post about ensuring we optimize for humans.
Back when I was a mid-level engineer at my first development job I learned a hard lesson that feedback is how you grow as a developer and make a better product. My very first professional PR was ~300 lines of code. It got 92 comments for changes and feedback.
Before the engineering lead left me the review he warned me that it would feel like a lot and to try not to take it personally, none of it was an indication that he didn't trust me or my work and was only meant to make the codebase better. What happened when the review landed and I saw the 92 comments? I got mad. I got sad. I got feelings that I was incapable of doing the job for real (hello Imposter Syndrome). I thought I couldn't code and why bother. Then I took a walk. Like away from the keyboard and computer and that review that made me upset.
Then I sat back down, put my headphones on, cranked some music and got back to work. The codebase became better for the review I got and the work that came out of it. I became a better developer. There was an art to getting a heavy review like that, one I didn't appreciate until much later in my career. The lead had the foresight to warn me in advance that it was a lot and not personal. I've tried to carry that same idea into later parts of my career, don't take code review personal. Sometimes I do better at this than others.
I had spent many hours on that code, actually I think it took me a week of work. I thought it was the best code I'd ever written in my life. Crushing to find out actually it wasn't (yet). But with that review it became even better than I had thought. It was more maintainable, extensible and testable than without that feedback.
This also wasn't even our application codebase for our main work, this was an internal only application. Did it truly matter that it was maintainable, and extensible? Probably not. But it instilled into me foundational experiences that I keep in mind every time I review code or am reviewed by someone else.
This was of course before LLMs came and took the tedious parts of crafting the syntax away from me (thankfully!), but cutting my teeth by hand was a great experience for me. Now days it's easy to throw up 300 lines of code before your breakfast warms in the microwave, it's cheaper and more expendable than ever.
But your humans that work on your systems and code are not any more expendable. We're still constrained by our lives, and limited hours and attention spans.
We're optimizing for the wrong thing with heavy LLM adoption. We should be adopting and using our tools that make every single human on our teams easier to spend less time on things. Prioritizing where the human token spend is worthwhile (coming up with new ideas, communicating those ideas to people comes to mind).
Perhaps you or a teammate on your team finds it easier to grok a Pull Request to review by using an LLM to process it, then you can build up that context and ask the LLM questions before you ask the code author's answer. What I do not think is an optimized use case is using an LLM to post 10-30 comments that are verbose on your behalf for a code review.
When an LLM writes under your avatar the reader assumes things about your intentions that is lost behind the verbosity. Do you actually want this change or is this you commenting so that it doesn't get lost? If the LLM writes 300+ words on a single chunk of code, is it really important to resolve? If you as the human write 300 words on a single chunk of code I'm for sure going to weigh that heavily, but it takes an LLM milliseconds to churn out 300 words.
I struggle because on a fundamental level, I think code is cheap enough now that it's a problem mostly solved good enough by LLMs with proper guidance and prompting. Changes people ask for are usually less painful than that first review I ever got was for me. But we loose that human connection and collaboration between having robots review PRs on our behalf.
Robots are great at catching things we don't always check, N+1s, missing test coverage and so forth, I will often time use an LLM or two pre-making the PR to review the code. Then I know at work for example there's an automated reviewer bot that runs as well. It checks different things, so sometimes things come up and I fix them. Then at that point after I fully reviewed the code, tested it and whatnot it's ready for my teammate's time.
My hope and expectation is that while I leveraged LLMs to help me make the code, and a teammate might use an LLM to understand changes... that there is some level of human collaboration between me throwing up a PR and then it getting approved/merged.
Why do we need this PR? Why in this way and not say approach #2? What things did you learn during development/testing that shaped this PR? Syntax is cheap, use a class here, use a method there whatever, add some tests, blah blah. No, fuck that. Tell me the meat of the learnings from this work, tell me what led you down this path instead of another. I can use my eyes behind my head and read the files changed. I don't need an LLM description to do that for me.
Otherwise like what's the point? If your LLM robot is improved over say a team reviewer bot, feed those improvements back to the team reviewer! The kind of comments an LLM is identifying and posting about are problems within the wheelhouse automated reviews should just handle.
Save your time for the stuff that actually matters, is this going to make phase 3 of a project harder? Why? Do we need to change scope on this or another project because of it? Perhaps you have actual experience with writing code in this area I'm working on. I want to hear from you, not tokens regurgitated by an LLM.
Jamie Lawrence, CTO at Podia (hi!) wrote a post today about "The Point". It's a great read. This is my take on his post, not an LLM etc. He wants to hear from YOU if you're asked something, not an LLM, not some regurgitation of the internet. Use an LLM on code or 3d model generation or something, but like spend the actual time to write your own documents, add that human touch.
I am sure this very blog post you're on has grammatical issues, heck maybe even something is misspelled. That's great, it's uncharacteristically human. There's an inhuman rhythm to LLM generated text that makes it painful to read hours a day of. I want to read your thoughts when you review code, when you write a PR description, when you write a document in Confluence or whatnot. It's OK to not answer everything anyone might ever have a question about in a comment on GitHub. Seek that human connection and optimize for those opportunities vs optimizing for an uber LLM efficient workplace.
✌️ That's all for now
Andrea