Code that does not work is just text.
No the spell just fizzled. In my experience it happens far less often if you start with an Abra kabara and end it with an Alakazam!
Yeah, the Abra kabara init and Alakazam cleanup are an important part, specially until you have become good enough to configure your own init.
There is an alternative init, Abra Kadabra, which automatically adds a cleanup and some general fixes when it detects the end of the spell.
Conversely, code that works is also text
Code that works is also just text.
Text that’s not code might also work.
deleted by creator
The image is taken from Zhihu, a Chinese Quora-like site.
The prompt is talking about give a design of a certain app, and the response seems to talk about some suggested pages. So it doesn’t seem to reflect the text.
But this in general aligns with my experience coding with llm. I was trying to upgrade my eslint from 8 to 9, and ask chatgpt to convert my eslint file, and it proceed to spit out complete garbage.
I thought this would be a good task for llm because eslint config is very common and well-documented, and the transformation is very mechanical, but it just cannot do it. So I proceed to read the documents and finished the migration in a couple hour…
I wouldn’t say it’s accurate that this was a “mechanical” upgrade, having done it a few times. They even have a migration tool which you’d think could fully do the upgrade but out of the probably 4-5 projects I’ve upgraded, the migration tool always produced a config that errored and needed several obscure manual changes to get working. All that to say it seems like a particularly bad candidate for llms
Then I am quite confused what LLM is supposed to help me with. I am not a programmer, and I am certainly not a TypeScript programmer. This is why I postponed my eslint upgrade for half a year, since I don’t have a lot of experience in TypeScript, besides one project in my college webdev class.
So if I can sit down for a couple hour to port my rather simple eslint config, which arguably is the most mechanical task I have seen in my limited programming experience, and LLM produce anything close to correct. Then I am rather confused what “real programmers” would use it for…
People here say boilerplate code, but honestly I don’t quite recall the last time I need to write a lot of boilerplate code.
I have also tried to use llm to debug SELinux and docker container on my homelab; unfortunately, it is absolutely useless in that as well.
With all due respect, how can you weigh in on programming so confidently when you admit to not being a programmer?
People tend to despise or evangelize LLMs. To me, github copilot has a decent amount of utility. I only use the auto-complete feature which does things like save me from typing 2-5 predictable lines of code that devs tend to type all the time. Instead of typing it all, I press tab. It’s just a time saver. I have never used it like “write me a script or a function that does x” like some people do. I am not interested in that as it seems like a sad crutch that I’d need to customize so much anyway that I may as well skip that step.
Having said that, I’m noticing the copilot autocomplete seems to be getting worst over time. I’m not sure why it worsening, but if it ever feels not worth it anymore I’ll drop it, no harm no foul. The binary thinkers tend to think you’re either a good dev who despises all forms of AI or you’re an idiot who tries to have a robot write all your code for you. As a dev for the past 20 years, I see no reason to choose between those two opposites. It can be useful in some contexts.
PS. did you try the eslint 8 -> 9 migration tool? If your config was simple enough for it, it likely would’ve done all or almost all the work for you… It fully didn’t work for me. I had to resolve several errors, because I tend to add several custom plugins, presets, and rules that differ across projects.
Sorry, the language my original post might seem confrontational, but that is not my intension; I m trying to find value in LLM, since people are excited for it.
I am not a professional programmer nor do I program any industrial sized project at the moment. I am a computer scientist, and my current research project do not involve much programming. But I do teach programming to undergrad and master students, so I want to understand what is a good usecase for this technology, and when can I expect it to be helpful.
Indeed, I am frustrated by this technology, and that might shifted my language further than I intended to. When everyone is promoting this as a magically helpful tool for CS and math, yet I fail to see any good applications for either in my work, despite going back to it every couple month or so.
I did try @eslint/migrate-config, unfortunately it added a good amount of bloat and ends up not working.
So I just gived up and read the doc.
Gotcha. No worries. I figured you were coming in good faith but wasn’t certain. Who is pushing llm’s for programming that hard? In my bubble, which often includes Lemmy, most people HATE them for all uses. I get that tech bros and linked in crazies probably push this tech for coding a lot, but outside of that, most devs I know IRL either are lukewarm or dislike llm’s for dev work.
Ai code is specifically annoying because it looks like it would work, but its just plausible bullshit.
And that’s what happens when you spend a trillion dollars on an autocomplete: amazing at making things look like whatever it’s imitating, but with zero understanding of why the original looked that way.
Well I’ve got the name for my autobiography now.
“Specifically Annoying” or “Plausible Bullshit”? I’d buy the latter.
This is where I got the term from: https://craphound.com/news/2023/09/17/plausible-sentence-generators/
To be fair, if I wrote 3000 new lines of code in one shot, it probably wouldn’t run either.
LLMs are good for simple bits of logic under around 200 lines of code, or things that are strictly boilerplate. People who are trying to force it to do things beyond that are just being silly.
Practically all LLMs aren’t good for any logic. Try to play ASCII tic tac toe against it. All GPT models lost against my four year old niece and I wouldn’t trust her writing production code 🤣
Once a single model (doesn’t have to be a LLM) can beat Stockfish in chess, AlphaGo in Go, my niece in tic tac toe and can one-shot (on the surface, scratch-pad allowed) a Rust program that compiles and works, than we can start thinking about replacing engineers.
Just take a look at the dotnet runtime source code where Microsoft employees currently try to work with copilot, which writes PRs with errors like forgetting to add files to projects. Write code that doesn’t compile, fix symptoms instead of underlying problems, etc. (just take a look yourself).
I don’t say that AI (especially AGI) can’t replace humans. It definitely can and will, it’s just a matter of time, but state of the Art LLMs are basically just extremely good “search engines” or interactive versions of “stack overflow” but not good enough to do real “thinking tasks”.
Cherry picking the things it doesn’t do well is fine, but you shouldn’t ignore the fact that it DOES do some things easily also.
Like all tools, use them for what they’re good at.
I don’t think it’s cherry picking. Why would I trust a tool with way more complex logic, when it can’t even prevent three crosses in a row? Writing pretty much any software that does more than render a few buttons typically requires a lot of planning and thinking and those models clearly don’t have the capability to plan and think when they lose tic tac toe games.
Why would I trust a drill press when it can’t even cut a board in half?
A drill press (or the inventors) don’t claim that it can do that, but with LLMs they claim to replace humans on a lot of thinking tasks. They even brag with test benchmarks, claim Bachelor, Master and Phd level intelligence, call them “reasoning” models, but still fail to beat my niece in tic tac toe, which by the way doesn’t have a PhD in anything 🤣
LLMs are typically good in things that happened a lot during training. If you are writing software there certainly are things which the LLM saw a lot of during training. But this actually is the biggest problem, it will happily generate code that might look ok, even during PR review but might blow up in your face a few weeks later.
If they can’t handle things they even saw during training (but sparsely, like tic tac toe) it wouldn’t be able to produce code you should use in production. I wouldn’t trust any junior dev that doesn’t set their O right next to the two Xs.
Sure, the marketing of LLMs is wildly overstated. I would never argue otherwise. This is entirely a red herring, however.
I’m saying you should use the tools for what they’re good at, and don’t use them for what they’re bad at. I don’t see why this is controversial at all. You can personally decide that they are good for nothing. Great! Nobody is forcing you to use AI in your work. (Though if they are, you should find a new employer.)
Totally agree with that and I don’t think anybody would see that as controversial. LLMs are actually good in a lot of things, but not thinking and typically not if you are an expert. That’s why LLMs know more about the anatomy of humans than I do, but probably not more than most people with a medical degree.
It’s futile even trying to highlight the things LLMs do very well as Lemmy is incredibly biased against them.
I can’t speak for Lemmy but I’m personally not against LLMs and also use them on a regular basis. As Pennomi said (and I totally agree with that) LLMs are a tool and we should use that tool for things it’s good for. But “thinking” is not one of the things LLMs are good at. And software engineering requires a ton of thinking. Of course there are things (boilerplate, etc.) where no real thinking is required, but non-AI tools like code completion/intellisense, macros, code snippets/templates can help with that and never was I bottle-necked by my typing speed when writing software.
It was always the time I needed to plan the structure of the software, design good and correct abstractions and the overall architecture. Exactly the things LLMs can’t do.
Copilot even fails to stick to coding style from the same file, just because it saw a different style more often during training.
“I’m not again LLMs I just never say anything useful about them and constantly point out how I can’t use them.” The other guy is right and you just prove his point.
Perhaps 5 LOC. Maybe 3. And even then I’ll analyze every single character in wrote. And then I will in fact find bugs. Most often it hallucinates some functions that would be fantastic to use - if they existed.
My guess is what’s going on is there’s tons of psuedo code out there that looks like it’s a real language but has functions that don’t exist as placeholders and the LLM noticed the pattern to the point where it just makes up functions, not realizing they need to be implemented (because LLMs don’t realize things but just pattern match very complex patterns).
I am on you with this one. It is also very helpful in argument heavy libraries like plotly. If I ask a simple question like “in plotly how do I do this and that to the xaxis” etc it generally gives correct answers, saving me having to do internet research for 5-10 minutes or read documentations for functions with 1000 inputs. I even managed to get it to render a simple scene of cloud of points with some interactivity in 3js after about 30 minutes of back and forth. Not knowing much javascript, that would take me at least a couple hours. So yeah it can be useful as an assistant to someone who already knows coding (so the person can vet and debug the code).
Though if you weigh pros and cons of how LLMs are used (tons of fake internet garbage, tons of energy used, very convincing disinformation bots), I am not convinced benefits are worth the damages.
Why do you want AI to save you for learning and understanding the tools you use?
If you do it through AI you can still learn. After all I go through the code to understand what is going on. And for not so complex tasks LLMs are good at commenting the code (though it can bullshit from time to time so you have to approach it critically).
But anyways the stuff I ask LLMs are generally just one off tasks. If I need to use something more frequently, I do prefer reading stuff for more in depth understanding.
All programs can be written with on less line of code. All programs have at least one bug.
By the logical consequences of these axioms every program can be reduced to one line of code - that doesn’t work.
One day AI will get there.
On one line of code you say?
*search & replaces all line breaks with spaces*
Fired for not writing the quota number of lines even junior devs manage to hit.
All programs can be written with on less line of code. All programs have at least one bug.
The humble “Hello world” would like a word.
You can fit an awful lot of Perl into one line too if you minimize it. It’ll be completely unreadable to most anyone, but it’ll run
The ideal code is no code at all
Big Beautiful Code
Watching the serious people trying to use AI to code gives me the same feeling as the cybertruck people exploring the limits of their car. XD
“It’s terrible and I should hate it, but gosh it it isn’t just so cool”
I wish i could get so excited over disappointing garbage
You definitely could use AI to code, the catch is you need to know how to code first.
I use AI to write code for mundane tasks all the time. I also review and integrate the code myself.
The AI code my “expert in a related but otherwise not helpful field” coworker writes helps me have a lot of extra work to do!
It’s useful if you just don’t do…That. it’s just a new fancy search engin, it’s a bit better than going to stack overflow, it can do good stuff if you go small.
Just don’t do whatever this post suggested of doing…
deleted by creator
This has beeny experience as well. It keeps emphasizing “beauty” and keeps missing “correctness”
Ctrl+A + Del.
So clean.
Write tests and run them, reiterate until all tests pass.
Bogosort with extra steps
This weekend I successfully used Claude to add three features in a Rust utility I had wanted for a couple years. I had opened issue requests, but no else volunteered. I had tried learning Rust, Wayland and GTK to do it myself, but the docs at the time weren’t great and the learning curve was steep. But Claude figured it all out pretty quick.
Did the generated code get merged? I’d be curious to see the PRs
The lead dev is not available this summer to review, but you can review here: https://github.com/edzdez/sway-easyfocus/pull/22
It’s not great that four changes are rolled into a single PR, but that’s my issue not Claude’s because they were related and I wanted to test them all at once.
This is interesting, I would be quite impressed if this PR got merged without additional changes.
I am genuinely curious and no judgement at all, since you mentioned that you are not a rust/GTK expert, are you able to read and and have a decent understanding of the output code?
For example, in the
sway.rsfile, you uncommented a piece of code about floating nodes inget_all_windowsfunction, do you know why it is uncommented? (again, not trying to judge; it is a genuine question. I also don’t know rust or GTK, just curious.This is interesting, I would be quite impressed if this PR got merged without additional changes.
We’ll see. Whether it gets merged in any form, it’s still a big win for me because I finally was able to get some changes implemented that I had been wanting for a couple years.
are you able to read and and have a decent understanding of the output code?
Yes. I know other coding languages and CSS. Sometimes Claude generated code that was correct but I thought it was awkward or poor, so I had it revise. For example, I wanted to handle a boolean case and it added three booleans and a function for that. I said no, you can use a single boolean for all that. Another time it duplicated a bunch of code for the single and multi-monitor cases and I had it consolidate it.
In one case, It got stuck debugging and I was able to help isolate where the error was through testing. Once I suggested where to look harder, it was able to find a subtle issue that I couldn’t spot myself. The labels were appearing far too small at one point, but I couldn’t see that Claude had changed any code that should affect the label size. It turned out two data structures hadn’t been merged correctly, so that default values weren’t getting overridden correctly. It was the sort of issue I could see a human dev introducing on the first pass.
do you know why it is uncommented?
Yes, that’s the fix for supporting floating windows. The author reported that previously there was a problem with the z-index of the labels on these windows, so that’s apparently why it was implemented but commented out. But it seems due to other changes, that problem no longer exists. I was able to test that labels on floating windows now work correctly.
Through the process, I also became more familiar with Rust tooling and Rust itself.
Thank you! This is very helpful.
Holy shit someone on here that know how to use them. Surprised you haven’t been downvoted into oblivion yet.
Did it try to blackmail him if he didn’t use the new code?
cant wait to see “we use AI agents to generate well structured non-functioning code” with off centered everything and non working embeds on the website
Honest question: I haven’t used AI much. Are there any AIs or IDEs that can reliably rename a variable across all instances in a medium sized Python project? I don’t mean easy stuff that an editor can do (e.g. rename QQQ in all instances and get lucky that there are no conflicts). I mean be able to differentiate between local and/or library variables so it doesn’t change them, only the correct versions.
Not reliably, no. Python is too dynamic to do that kind of thing without solving general program equivalence which is undecidable.
Use a static language, problem solved.
I’m going to laugh in Java, where this has always been possible and reliable. Not like ai reliable, but expert reliable. Because of static types.
IntelliJ IDEA, if it knows it is the same variable, it will rename it. Usually works in a non fucked up codebase that uses
evalor some obscure constructs like saving a variable name into a variable as a string and dynamically invoking it.For the most part “Rename symbol” in VSCode will work well. But it’s limited by scope.
Yeah, I’m looking for something that would understand the operation (? insert correct term here) of the language well enough to rename intelligently.
I use pycharm for this and in general it does a great job. At work we’ve got some massive repos and it’ll handle it fine.
The “find” tab shows where it’ll make changes and you can click “don’t change anything in this directory”
Yes, all of JetBrains’ tools handle project-wide renames practically perfectly, even in weirder things like Angular projects where templates may reference variables.
Just be carerul when refactoring variable names in doc comments, I’ve seen some weird stuff happen there
Itellij is actually pretty good at this. Besides that, cursor or windsurf should be able to. I was using cursor for a while and when I needed to reactor something, it was pretty good at picking that up. It kept crashing on me though, so I am now trying windsurf and some other options. I am missing the auto complete features in cursor though as I would use this all the time to fill out boilerplate stuff as I write.
The one key difference in cursor and windsurf when compared to other products is that it will look at the entire context again for any changes or at least a little bit of it. You make a change, it looks if it needs to make changes elsewhere.
I still don’t trust AI to do much though, but it’s an excellent helper
Find and Replace?
that will catch too many false positives
most IDEs are pretty decent at it if you configure them correctly. I used intelliJ and it knows the difference. use the refactor feature and it’ll crawl references, not just rename all instances.
