Lots of people swear by large-language model (LLM) AIs for writing code. Lots of people swear at them. Still others may be planning to exploit their peculiarities, according to [Joe Spracklen] and other researchers at USTA. At least, the researchers have found a potential exploit in ‘vibe coding’.

Everyone who has used an LLM knows they have a propensity to “hallucinate”– that is, to go off the rails and create plausible-sounding gibberish. When you’re vibe coding, that gibberish is likely to make it into your program. Normally, that just means errors. If you are working in an environment that uses a package manager, however (like npm in Node.js, or PiPy in Python, CRAN in R-studio) that plausible-sounding nonsense code may end up calling for a fake package.

A clever attacker might be able to determine what sort of false packages the LLM is hallucinating, and inject them as a vector for malicious code. It’s more likely than you think– while CodeLlama was the worst offender, the most accurate model tested (ChatGPT4) still generated these false packages at a rate of over 5%. The researchers were able to come up with a number of mitigation strategies in their full paper, but this is a sobering reminder that an AI cannot take responsibility. Ultimately it is up to us, the programmers, to ensure the integrity and security of our code, and of the libraries we include in it.

We just had a rollicking discussion of vibe coding, which some of you seemed quite taken with. Others agreed that ChatGPT is the worst summer intern ever.  Love it or hate it, it’s likely this won’t be the last time we hear of security concerns brought up by this new method of programming.

Special thanks to [Wolfgang Friedrich] for sending this into our tip line.

Vibe coding is the buzzword of the moment. What is it? The practice of writing software by describing the problem to an AI large language model and using the code it generates. It’s not quite as simple as just letting the AI do your work for you because the developer is supposed to spend time honing and testing the result, and its proponents claim it gives a much more interactive and less tedious coding experience. Here at Hackaday, we are pleased to see the rest of the world catch up, because back in 2023, we were the first mainstream hardware hacking news website to embrace it, to deal with a breakfast-related emergency.

Jokes aside, though, the fad for vibe coding is something which should be taken seriously, because it’s seemingly being used in enough places that vibe coded software will inevitably affect our lives.  So here’s the Ask Hackaday: is this a clever and useful tool for making better software more quickly, or a dangerous tool for creating software nobody quite understands, containing bugs which could cause a disaster?

Our approach to writing software has always been one of incrementally building something from the ground up, which satisfies the need. Readers will know that feeling of being in touch with how a project works at all levels, with a nose for immediately diagnosing any problems that might occur. If an AI writes the code for us, the feeling is that we might lose that connection, and inevitably this will lead to less experienced coders quickly getting out of their depth. Is this pessimism, or the grizzled voice of experience? We’d love to know your views in the comments. Are our new AI overlords the new senior developers? Or are they the worst summer interns ever?

What a long, strange trip it’s been for NASA astronauts Suni Williams and Bruce Wilmore, who finally completed their eight-day jaunt to space after 289 days. The duo returned to Earth from the ISS on Tuesday along with two other returning astronauts in a picture-perfect splashdown, complete with a dolphin-welcoming committee. For the benefit of those living under rocks these past nine months, Williams and Wilmore slipped the surly bonds way back in June on the first crewed test flight of the Boeing Starliner, bound for a short stay on the ISS before a planned return in the same spacecraft. Alas, all did not go to plan as their ride developed some mechanical difficulties on the way upstairs, and so rather than risk their lives on a return in a questionable capsule, NASA had them cool their heels for a couple of months while Starliner headed home without them.

There’s been a lot of talk about how Butch and Suni were “stranded,” but that doesn’t seem fair to us. Sure, their stay on the ISS was unplanned, or at least it wasn’t Plan A; we’re sure this is always a contingency NASA allows for when planning missions. Also unfortunate is the fact that they didn’t get paid overtime for the stay, not that you’d expect they would. But on the other hand, if you’re going to get stuck on a work trip, it might as well be at the world’s most exclusive and expensive resort.

Speaking of space, while it’s statistically unlikely that anyone reading this will ever get there, you can still get a little taste of what space travel is like if you’re willing to give up ten days of your life to lie in a waterbed. What’s more, the European Space Agency will pay you 5,000 euros to do it. The experiment is part of the ESA’s Vivaldi III campaign, an exploration of the effects of extended spaceflight on the human body. The “waterbed” thing is a little misleading, though; since the setup is designed to simulate the posture the body takes in microgravity, they use a tank of water (heated, we hope) with a waterproof cover to submerge volunteers up to their torso. This neutral body posture looks pretty comfortable if you’re sleeping in space, but we tend to think it’d get annoying pretty quickly down here. Especially for potty breaks, which aren’t done astronaut-style but rather by being transferred to a trolley which lets you do your business without breaking from the neutral posture. Still, 5,000 euros is 5,000 euros.

Bad news for the meme-making community, as it appears AI might be coming for you, too. A recent study found that LLMs like ChatGPT can meme better than humans, at least under certain conditions. To come to that conclusion, researchers used some pretty dank meme templates and pitted humans against ChatGPT-4o to come up with meme-worthy captions. They also had a different group of humans collaborate with the LLM to come up with meme captions, which for practical purposes probably means the humans let the chatbot do the heavy lifting and just filtered out the real stinkers. When they showed the memes to crowdsourced participants to rate them on humor, creativity, and shareability, they found that the LLM consistently produced memes that scored higher across all three categories. This makes sense when you think about it; the whole job of an LLM is to look at a bunch of words and come up with a consensus on what the next word should be. Happily, the funniest memes were written by humans, and the human-LLM collaborations were judged more creative and shareable. So we’ve got that going for us, which is good.

We noted the passing of quite a few surplus electronics shops in this space before, and the closing of each of them, understandable as they may, marks the end of an era. But we recently learned about one surplus outfit that’s still going strong. Best Electronics, which specializes in Atari retrocomputing, has been going strong for over 40 years, a neat trick when Atari itself went bankrupt over 30 years ago. While they appear to have a lot of new old stock bits and bobs — they’re said to have acquired “thousands and thousands” of pallets of Atari goods from their Sunnyvale warehouse when the company folded — they also claim to spend a lot of money on engineering development. Their online presence is delightfully Web 1.0, making it pretty hard to sort through, but we think that development is mainly upgraded PCBs for things like joysticks and keyboards. Whatever they’re doing, they should just keep on doing it.

And finally, have you ever seen a knitted breadboard? Now you have, and while it’s of no practical value, we still love it. Alanna Okun made it for the ITP Stupid Hackathon at NYU back in February. There aren’t any instructions or build docs, so it’s not clear how it works, but from the photos we’d guess there’s either conductive yarn or solid copper wire knitted into the pattern to serve as bus bars.

“The brickings will continue until the printer sales improve!” This whole printer-bricking thing seems to be getting out of hand with the news this week that a firmware update caused certain HP printers to go into permanent paper-saver mode. The update was sent to LaserJet MFP M232-M237 models (opens printer menu; checks print queue name; “Phew!) on March 4, and was listed as covering a few “general improvements and bug fixes,” none of which seem very critical. Still, some users reported not being able to print at all after the update, with an error message suggesting printing was being blocked thanks to non-OEM toner. This sounds somewhat similar to the bricked Brother printers we reported on last week (third paragraph).

The trouble is, some users are reporting the problem even if they had genuine HP toner installed. Disturbingly, HP support seems to be fine with this, saying that older HP toner “may no longer be recognized due to new security measures.” Well, there’s your problem, lady! The fix, of course, is to buy yet more genuine HP toner, even if your current cartridge still has plenty of life left in it. That’s a pretty deplorable attitude on HP’s part, and more than enough reason to disable automatic firmware updates, or better yet, just disconnect your printer from the Internet altogether.

Here’s a pro-tip for all you frustrated coders out there: no matter how hard the job gets, planting a logic bomb in your code is probably not the right way to go. That’s the lesson that one Davis Lu learned after being convicted of “causing intentional damage to protected computers” thanks to malicious code he planted in his employer’s system. Apparently not optimistic about his future prospects with Eaton Corp. back in 2018, Lu started adding code designed to run a series of infinite loops to delete user profiles. He also went for the nuclear option, adding code to shut the whole system down should it fail to find an Active Directory entry for him. That code was apparently triggered on the day he was fired in 2019, causing global problems for his former employer. Look, we’ve all been there; coding is often lonely work, and it’s easy to fantasize about coding up something like this and watching them squirm once they fire you. But if it gets that bad, you should probably put that effort into finding a new gig.

Then again, maybe the reason you’re dissatisfied with your coding job is that you know some smart-ass LLM is out there waiting to tell you that you don’t know how to code. That’s what happened to one newbie Cursor user who tried to get help writing some video game code from the AI code editor. The LLM spat back about 750 lines of code but refused to reveal the rest, and when he asked to explain why, it suggested that he should develop the logic himself so that he’d be able to understand and maintain the code, and that “Generating code for others can lead to dependency and reduced learning opportunities.” True enough, but do we really need our AI tools to cop an attitude?

And finally, if you’re anything like us, you’re really going to love this walking tour of a container ship’s mechanical spaces. The ship isn’t named, but a little sleuthing suggests it’s one of the Gülsün-class ships built for MSC in 2019, possibly the MSC Mina, but that’s just a guess. This 400-meter monster can carry 23,656 twenty-foot equivalent units, and everything about it is big. Mercifully, the tour isn’t narrated, not that it would have been possible, thanks to the screaming equipment in the engine room. There are captions, though, so you’ll at least have some idea of what you’re looking at in the immaculately clean and cavernously huge spaces. Seriously, the main engine room has to have at least a dozen floors; being on the engineering crew must mean getting your steps in every day. The most striking thing about the tour was that not a single other human being was visible during the entire hour. We suppose that’s just a testament to how automated modern vessels have become, but it still had a wonderfully creepy liminal feeling to it. Enjoy!

[Simon Willison] has put together a list of how, exactly, one goes about using a large language models (LLM) to help write code. If you have wondered just what the workflow and techniques look like, give it a read. It’s full of examples, strategies, and useful tips for effectively using AI assistants like ChatGPT, Claude, and others to do useful programming work.

It’s a very practical document, with [Simon] emphasizing realistic expectations and the importance of managing context (both in terms of giving the LLM direction, as well as the model’s context in terms of being mindful of how much the LLM can fit in its ‘head’ at once.) It is useful to picture an LLM as a capable and obedient but over-confident programming intern or assistant, albeit one that never gets bored or annoyed. Useful work can be done, but testing is crucial and human oversight simply cannot be automated away.

Even if one has no interest in using LLMs to help in writing production code, there’s still a lot of useful work they can do to speed up the process of software development in general, especially when learning. They can help research options, interactively explore unfamiliar codebases, or prototype ideas quickly. [Simon] provides useful strategies for all these, and more.

If you have wondered how exactly glorified chatbots can meaningfully help with software development, [Simon]’s writeup hopefully gives you some new ideas. And if this is is all leaving you curious about how exactly LLMs work, in the time it takes to enjoy a warm coffee you can learn how they do what they do, no math required.

Ho-hum — another week, another high-profile bricking. In a move anyone could see coming, Humane has announced that their pricey AI Pin widgets will cease to work in any meaningful way as of noon on February 28. The company made a splash when it launched its wearable assistant in April of 2024, and from an engineering point of view, it was pretty cool. Meant to be worn on one’s shirt, it had a little bit of a Star Trek: The Next Generation comm badge vibe as the primary UI was accessed through tapping the front of the thing. It also had a display that projected information onto your hand, plus the usual array of sensors and cameras which no doubt provided a rich stream of user data. Somehow, though, Humane wasn’t able to make the numbers work out, and as a result they’ll be shutting down their servers at the end of the month, with refunds offered only to users who bought their AI Pins in the last 90 days.

How exactly Humane thought that offering what amounts to a civilian badge cam was going to be a viable business model is a bit of a mystery. Were people really going to be OK walking into a meeting where Pin-wearing coworkers could be recording everything they say? Wouldn’t wearing a device like that in a gym locker room cause a stir? Sure, the AI Pin was a little less obtrusive than something like the Google Glass — not to mention a lot less goofy — but all wearables seem to suffer the same basic problem: they’re too obvious. About the only one that comes close to passing that hurdle is the Meta Ray-Ban smart glasses, and those still have the problem of obvious cameras built into their chunky frames. Plus, who can wear Ray-Bans all the time without looking like a tool?

Good news for everyone worried about a world being run by LLMs and chatbots. It looks like all we’re going to have to do is wait them out, if a study finding that older LLMs are already showing signs of cognitive decline pans out. To come to that conclusion, researchers gave the Montreal Cognitive Assessment test to a bunch of different chatbots. The test uses simple questions to screen for early signs of impairment; some of the questions seem like something from a field sobriety test, and for good reason. Alas for the tested chatbots, the general trend was that the older the model, the poorer they did on the test. The obvious objection here is that the researchers aren’t comparing each model’s current score with results from when the model was “younger,” but that’s pretty much what happens when the test is used for humans.

You’ve got to feel sorry for astronomers. Between light pollution cluttering up the sky and an explosion in radio frequency interference, astronomers face observational challenges across the spectrum. These challenges are why astronomers prize areas like dark sky reserves, where light pollution is kept to a minimum, and radio quiet zones, which do the same for the RF part of the spectrum. Still, it’s a busy world, and noise always seems to find a way to leak into these zones. A case in point is the recent discovery that TV signals that had been plaguing the Murchison Wide-field Array in Western Australia for five years were actually bouncing off airplanes. The MWA is in a designated radio quiet zone, so astronomers were perplexed until someone had the bright idea to use the array’s beam-forming capabilities to trace the signal back to its source. The astronomers plan to use the method to identify and exclude other RFI getting into their quiet zone, both from terrestrial sources and from the many satellites whizzing overhead.

And finally, most of us are more comfortable posting our successes online than our failures, and for obvious reasons. Everyone loves a winner, after all, and admitting our failures publicly can be difficult. But Daniel Dakhno finds value in his failures, to the point where he’s devoted a special section of his project portfolio to them. They’re right there at the bottom of the page for anyone to see, meticulously organized by project type and failure mode. Each failure assessment includes an estimate of the time it took; importantly, Daniel characterizes this as “time invested” rather than “time wasted.” When you fall down, you should pick something up, right?

Large language models (LLMs) are all the rage in the generative AI world these days, with the truly large ones like GPT, LLaMA, and others using tens or even hundreds of billions of parameters to churn out their text-based responses. These typically require glacier-melting amounts of computing hardware, but the “large” in “large language models” doesn’t really need to be that big for there to be a functional, useful model. LLMs designed for limited hardware or consumer-grade PCs are available now as well, but [Binh] wanted something even smaller and more portable, so he put an LLM on a USB stick.

This USB stick isn’t just a jump drive with a bit of memory on it, though. Inside the custom 3D printed case is a Raspberry Pi Zero W running llama.cpp, a lightweight, high-performance version of LLaMA. Getting it on this Pi wasn’t straightforward at all, though, as the latest version of llama.cpp is meant for ARMv8 and this particular Pi was running the ARMv6 instruction set. That meant that [Binh] needed to change the source code to remove the optimizations for the more modern ARM machines, but with a week’s worth of effort spent on it he finally got the model on the older Raspberry Pi.

Getting the model to run was just one part of this project. The rest of the build was ensuring that the LLM could run on any computer without drivers and be relatively simple to use. By setting up the USB device as a composite device which presents a filesystem to the host computer, all a user has to do to interact with the LLM is to create an empty text file with a filename, and the LLM will automatically fill the file with generated text. While it’s not blindingly fast, [Binh] believes this is the first plug-and-play USB-based LLM, and we’d have to agree. It’s not the least powerful computer to ever run an LLM, though. That honor goes to this project which is able to cram one on an ESP32.

Giskard is an open-source AI model quality testing tool that helps data scientists and engineers build safer, more reliable AI systems. The platform was built by AI engineers for AI engineers. It’s completely open source and designed to help teams and developers build more robust, trustworthy AI models. To use the platform, you can get […]

Source

Faraday.dev lets you easily run open-source LLMs (chatbots) on your computer. Once you’ve got the program and AI models installed, no internet connection is required to use and interact with the AI LLMs. Faraday.dev supports a wide range of LLaMA-based models, including WizardLM, GPT4-x-Alpaca, Vicuna, Koala, Open Assistant, PygmalionAI, and more. You have the option […]

Source

Oobabooga is an open-source Gradio web UI for large language models that provides three user-friendly modes for chatting with LLMs: a default two-column view, a notebook-style interface, and a chat interface. This flexibility allows you to interact with the AI models in a way that best suits your needs, whether it’s for writing, analysis, question-answering, […]

Source

Code Llama is a suite of large language models released by Meta AI for generating and enhancing code. It includes foundation models for general coding, Python specializations, and models tailored for following instructions. Key features include state-of-the-art performance, code infilling, large context support up to 100K tokens, and zero-shot ability to follow instructions for programming […]

Source

ChatGLM-6B is an open-source, bilingual conversational AI LLM based on the General Language Model (GLM) framework. It has 6.2 billion parameters and can be deployed locally with only 6GB of GPU memory. This model allows for natural language processing in both Chinese and English, question answering, task-oriented dialogue, and easy integration via API and demo […]

Source

Perplexity AI is an AI chat and search engine that uses advanced technology to provide direct answers to your queries. It delivers accurate answers using large language models and even includes links to citations and related topics. It is available for free via web browser and also on mobile via the Apple App Store. Using […]

Source

Codestral is a powerful 22B parameter AI model from Mistral AI. This open-weight model is designed specifically for code generation across over 80 programming languages including Python, Java, C++, JavaScript and more. Codestral offers impressive performance, outperforming other models on benchmarks like HumanEval and RepoBench with its large 32k token context window. The model is […]

Source

Langtail is a platform that helps you develop and deploy LLM-powered applications faster. It provides tools for prompt engineering, testing, observability, and deployment – all in one place. You can collaborate with your team, iterate quickly, and get your LLM apps to production with confidence.

Source

Mistral AI is a large language model and chat assistant tool. You can access the chatbot via the Mitral website by clicking on “Talk to le Chat“, or if you prefer a local setup then you can download and run the model files on your own hardware. The creators of Mistral describe it as an […]

Source

Stability AI, the team behind the popular AI art tool Stable Diffusion, has announced the launch of its latest creation: StableLM, a suite of text-generating AI models designed to rival systems like OpenAI’s GPT-4 and ChatGPT. Available in “alpha” on GitHub and Hugging Face, StableLM can generate both code and text and has been trained […]

Source