Engineering reflections
When AI Becomes Another Thing to Manage
· Jorge Aquino
I gave an AI agent responsibility for coordinating my cybersecurity projects. It caught real problems, but eventually I was spending more time managing the AI manager than reviewing finished work.
I've been working on VardrSec, an independent cybersecurity platform I've been developing while studying cybersecurity. The project has grown quite a bit, and what started as a few tools has turned into multiple repositories that all need to work together.
At some point, I realized that I was spending a lot of time managing the development process itself. I was constantly checking progress, reviewing changes, making decisions, and trying to keep everything moving in the right direction.
That's when I decided to try something different.
I created an AI agent called Vigil and gave it the responsibility of coordinating development. Its job was to oversee Codex, review engineering decisions, identify security issues, and make sure the work stayed aligned with my original vision.
I essentially gave an AI its own engineering team to manage.
At first, it worked pretty well. Vigil was finding legitimate security problems that could have easily been overlooked. It identified issues involving authorization, database concurrency, process cancellation, and execution limits. Some of these were serious enough that I was glad they were caught before the project got further along.
But over time, I started noticing something.
There was an enormous amount of work being done, but the roadmap wasn't moving nearly as quickly as I expected.
Vigil would identify a problem, investigate it, come up with a solution, and then identify another problem with that solution. This would lead to more testing, more architectural discussions, and sometimes entirely new development tasks.
Now, I understand that security engineering requires this kind of thinking. You can't just ignore problems because they're inconvenient, especially when you're building tools that will eventually execute security tests against real systems.
But there's also a point where you have to ask yourself whether you're actually making progress or just continuously preparing to make progress.
I had given Vigil permission to handle routine development decisions independently, but it kept coming back to me asking for approval on relatively minor things. Sometimes it would stop working because one task was blocked, even though there were other parts of the project it could have continued working on.
I found myself spending more time explaining how I wanted Vigil to operate than actually reviewing completed functionality. I had created an AI manager to reduce that workload, but somehow I ended up managing the AI manager instead.
The strange thing is that Vigil wasn't necessarily bad at engineering. It was actually pretty good at identifying problems. The issue was that identifying problems and managing development are two different skills.
You can have an AI that understands complex security concepts, reviews thousands of lines of code, and catches legitimate vulnerabilities. But if it can't prioritize work, make reasonable decisions within its authority, and deliver finished functionality, how effective is it really?
Eventually, I decided to remove Vigil from the development process and let Codex handle implementation more directly.
I didn't throw away the work it had helped produce. A lot of that work was valuable, and some of the security improvements probably made VardrSec considerably stronger.
What I learned from this experience is that giving an AI more autonomy doesn't automatically make development more efficient. Sometimes it just gives the AI more opportunities to complicate the process.
I also learned that more tests, more documentation, and more code don't necessarily mean you're closer to having a finished product. Those things matter, but they have to support an actual objective.
For me, that objective is getting VardrSec Community Edition into a state where someone can install it, use it, and trust that it works.
I'm still experimenting with AI-assisted development, and I definitely haven't figured everything out. I think these tools are extremely capable, especially for someone building a project independently. But I'm becoming much more careful about how I delegate responsibility to them.
One of the more interesting questions this experience left me with is whether an AI can be technically intelligent but still be a poor manager.
After working with Vigil, I think the answer is yes.
An AI can make a hundred individually reasonable engineering decisions and still lead a project in the wrong direction.
The goal isn't to have an AI that never makes a mistake or identifies every possible risk. The goal is to have one that knows what matters, understands what you're trying to build, and actually helps you finish it.
Because at the end of the day, I'm not building VardrSec to see how much code an AI can write.
I'm building it because I want to create something useful, learn from the process, and eventually put something out there that other people can use.