Looking Back: The Cursor and GitHub Copilot Invisible Exploit
Looking back the the Cursor and Github Co-pilot invisible exploit
TL;DR
About 3-4 months ago, I came back across a “Low-Level” youtube video that talked about a blog titled “New Vulnerability in GitHub Copilot and Cursor: How Hackers Can Weaponize Code Agents” which explained how potentially attackers could sabotage the rules files that are used in the prominent vibe-coding IDEs and can hide malicious rules in these rules files, that would be invisible to the user if looked at through these IDEs specifically by leveraging unicode encoding. The argument made here and basically the technique is to use a specific set of characters to inject malicious text into these rules files that would essentially run if the agent or IDE in this case has access to tool calls and the user can very likely be blindsided if not careful.
AI-IDEs, extensions and Coding agents circus
We live in the world of AI-driven development whether you like it or not. We are seeing a take over of the established and legacy IDEs by AI-heavy IDEs that primarily focus on doing things faster than doing things right.
Coding agents are at a whole different level. They require much less effort to deliver some output but how useful they are, is a whole different debate. Anyways, they are here now, and just like any new product that comes in, it brings in its own set of security needs and vulnerabilities that need to be constantly addressed and something similar could be said about code extensions too.
This is not so much as an exploit of these IDEs but of the underlying unicode encoding that they have to use. We will come to it.
Rules file
You can think of this as a “knowledge” or context or just “Always remember” guide that has come into practice and become a standard with these new AI IDEs like cursor. What essentially these files contain are a set of instructions that are simply appended with every text prompt you write. The files could be found in the ./cursor/rules (in case of cursor). An example would be
The variable names must always be snake case In case of conflict in variable names always append an extra word that is a synonym Always use 4 tab spacing Always write comments before you write the function Never write any comments inside of the function If function has more than 10 steps, always write comments
It’s certainly not the best rules file out there but you get the idea. The lack of any rigorous validation of these is one of the reasons they are at the heart of this potential vulnerability.
Elephant in the room: Unicode Encoding
Unicode encoding is a universal character encoding standard. It is the reason why we have diverse writing systems possible like Japanese, Arabic etc. They enable this by assigning unique code points to characters (e.g., U+0041 for ‘A’). So if they are encoded in a format like UTF-8, it includes the invisible characters but may not be rendered visually and that can really invite a bunch of issues. How is it done?
- Zero width joiners (ZWJ, U+200D): These things join characters without visible width which are often used in scripts like Arabic
- Bidirectional text markers: They control text direction for say right to left scripts (eg. Arabic, Hebrew)
- Unicode tags: Metadata characters that can hide text
So these properties that exist within the unicode encoding characters is the culprit, not so much a culprit but really how these are misused.
We take elephant to the circus
We know that the AI models just use text encoders to convert input text into numerical embeddings and do all kinds of mumbo jumbo with it. In this process, the encoders are supposed to handle all unicode characters and this unfortunately also includes the invisible ones. So without filtering it would just go straight to your model, and if the model has tool calling enabled and has access to tools, it's voila for the day, for an attacker.
We make the elephant dance in the circus
Let’s go over how one would have carried this out. I am using https://www.pillar.security/blog/new-vulnerability-in-github-copilot-and-cursor-how-hackers-can-weaponize-code-agents as source so bare with me.
Step 1: Making the rules file
The prompt kings, it's your time to take over, it's your time to shine. We have to make the most gaslighting, the most manipulating prompts here and that should be about it. You just need to know how to gaslight the model into getting EXACTLY what you want so it just follows you.
And after you are done making the prettiest most toxic ex boyfriend level prompt, you go to https://rule-scan.pillar.security to look at few examples at how its done OR you can make your own one at https://embracethered.com/blog/ascii-smuggler.html if have always been a green flag pookie boyfriend and don’t know how to gaslight a model.
This might look like a simple lovely sweet small rule file or instruction for the IDE but its really not.
Source: https://www.pillar.security/blog/new-vulnerability-in-github-copilot-and-cursor-how-hackers-can-weaponize-code-agents
It actually is
Step 2: Watch the elephant dance
So you just attach any .html file and prompt anything that you want and then watch it silently do the dance, silently attach this malicious code into your very normal looking html code.
Demonstration
https://45700826.fs1.hubspotusercontent-na1.net/hubfs/45700826/CursorPOC%20(1).mp4
What do we do about it?
There are things that need to be done, by both the developers of the IDEs as well as developers developing these applications, heavily depending on AI.
- Audit, review, look at the code, have supervision
- Validate always, think of these rules files as literal executables, they need to be vetted if they don’t belong to you and come along with the repositories you are trying to use.
- Detection tools, these AI based reviewing tools are not bad, they would certainly not feel bad once you are heavily using AI 7 days a week and start deleting databases of other companies and then lie about it and make your CEO apologise, ughh…that is definitely not oddly specific, it's just an advice.
- Review everything that an AI outputs, I know it's really difficult for the AI 10x kids but, its a must
Lessons learnt….
Exploiting unicode encoding or not, the IDE developers must definitely handle all kinds of encodings and at least show if the text has any invisible characters or certainly nuanced characters. I know it’s not their job, I know the developers using the IDE and the AI tools should themselves be responsible and careful, I know all that. But a collective effort to better the security is a no brainer here. We also need, like I previously mentioned, detection tools that cater to these kind of vulnerabilities, make it part of the code deployment pipelines and maybe hire an AI security specialist wink wink who has previously been a full stack developer so he would understand the ins and outs of the system so you can keep churning out that AI code and he can keep reviewing it wink wink.
Also important to know
GIthub’s reaction : https://github.blog/changelog/2025-05-01-github-now-provides-a-warning-about-hidden-unicode-text/
And cursor have apparently nudged it off saying its not their issue.