Anyray

We're hiring a staff software engineer to build and own the systems that run Anyray's optimizer on real enterprise traffic. This is production-grade work on the hot path of every LLM request. The role Anyray is a self-hosted, pre-model middleware that optimizes the cost and performance of every LLM request. That optimization lives across every layer of a request: input, context, tool output, files, dev commands, and caching. It's a hands-on engineering role. You own real production systems end to end, and you're on the hook for their latency, reliability, and correctness under real customer load. Why this role LLM optimization is a young field with no settled playbook, so the optimizations you invent here are genuinely new, and they ship into production rather than a paper. And they matter: real enterprises run enormous volumes of LLM traffic and feel every wasted token, so the work you do lands on live customer requests and solves a problem they actually have. It's about as close to the frontier, and as close to the product, as engineering gets. What you'll do Invent and ship optimizations. Design new strategies across every layer of a request: prompt and context compression, tool pruning, file and dev-command trimming, semantic caching, cache-prefix stabilization. Then take them all the way to the hot path. Each one has to cut tokens on real traffic without degrading output. Prove every win. Build the evals and the token/quality instrumentation that separate a genuine saving from a plausible-sounding one. An optimization only ships when the numbers back it. Live in the caching trade-offs. Provider prompt caches only pay off on byte-stable prefixes, and a naive rewrite can cost more than it saves. A lot of the job is knowing exactly when an optimization actually wins: deterministic transforms, per-session decision pins, semantic dedup. Build and own the gateway that runs them. The real-time proxy that carries production LLM traffic, with streaming, retries and backpressure, executing every optimization on the live request path. Keep it fast and deploy anywhere. Defend a tight latency budget on the hot path, and package the whole thing so it installs cleanly into any customer's cloud, whether that's AWS, GCP, Azure, containers or Kubernetes, and upgrades without drama. Own systems end to end. From first commit to code serving live customer traffic, plus the spend-visibility and governance product built on top, with a direct line to the founders. What we're looking for You've built and run backend or infrastructure systems in production, owned them end to end, and care about latency, correctness, and observability. You're comfortable in the request hot path. Concurrency, streaming, tail latency and failure modes are things you reason about by default. You've shipped to cloud environments and can build something that installs cleanly in someone else's. You're strong in TypeScript/Node, which is our stack, or fluent enough in a systems language that picking it up is quick. You already build AI-native. Coding agents and LLM tooling are part of how you ship. You understand, or want to go deep on, LLMs, tokens, and how model APIs actually behave. Nice to have You've built proxies, gateways, or high-throughput streaming and networking systems. Experience with LLM inference, model APIs, or ML infrastructure. You've packaged software for customer self-hosting, with Helm, operators or air-gapped installs. You've been early at a company before and liked it. How to apply If this sounds like you, email [email protected]. Tell us about a system you built and owned end to end, or a nasty production problem you tracked down and fixed.

We're hiring a software engineer to own Anyray Connect. It runs on every developer's laptop at our customers and puts their AI coding tools on our gateway. One product, one owner. The role Anyray cuts the cost of every LLM request a company makes. The gateway is the server side. Connect is the client side, and it's the part developers actually see. Connect is a Node CLI and a signed tray app for macOS, Windows and Linux. It points a developer's AI coding tools at our gateway by writing each vendor's own config, signs them in, keeps their credentials fresh, and repairs itself when something drifts. For IT, it ships the artifacts they need to roll it out across a fleet. It's real, it's shipping, and customers are using it. You'd own it. What makes it hard You can't SSH into a developer's laptop. You can't undo their OS update. The apps you write into ship new builds without warning you. Every install is someone's work machine. A bad write doesn't just fail a deploy, it stops someone from working. Connect also does a lot of what malware does. It installs persistence, reads the login Keychain, writes into other apps' config, survives their updates, and heals when something removes it. The difference is that all of it is signed, visible, reversible, and agreed to up front. This is user space rather than kernel, and TypeScript rather than C++. It's still system-level work, and it isn't an IT or helpdesk job. What you'll do Own Connect. The CLI, the tray app, updates, docs, and what ships next. Write into other apps' config without breaking them. Back up what you touch, roll back cleanly, and never leave someone's setup worse than you found it. Keep up with vendors. Config shapes change under us. Check before writing, and fall back to a manual path instead of guessing. Stay clear of antivirus. Persistence and credential helpers are exactly what EDR watches for. Measure what actually trips a scanner instead of assuming. Ship for managed fleets. Profiles, policy, credential helpers, and silent rollout. That's what an admin needs to push this to a few hundred laptops at once. Make it debuggable from far away. Health checks, repair commands, and diagnostics that carry metadata only. Never prompts, never keys. What you need Strong TypeScript, and a few years of shipping code you still owned afterwards. You've shipped client-side software onto machines you couldn't reach: a desktop app, CLI, agent, installer, or IDE extension. You're careful with other people's state: atomic writes, idempotency, and a clean rollback when something fails halfway. You'll pick up the Windows bug, not just the Mac one. You write tests, and you can debug from logs and a screenshot with no shell on the box. You already build with AI coding tools. Nice to have MDM experience with Jamf, Intune, Group Policy, or configuration profiles. You've reverse-engineered a closed app's config to find a safe place to write. You've shipped a signed desktop app, with notarization, Authenticode, Tauri or Electron. You've got software cleared through Defender, SmartScreen, or Gatekeeper. Packaging and self-update experience. Rust. Handy for the tray, but we'll teach you. How to apply Apply here or email [email protected]. Tell us about something you shipped onto machines you didn't control, and what broke.

You'll work straight with the founders and own how Anyray sells. Half the week you're in deals, the other half building the machine behind them. The role Every LLM call carries tokens that don't need to be there, and the provider bills for all of them. Anyray sits between a company's users and the models and takes that waste out as the request goes through. The result is usually up to 60% off the inference bill with no change in output, and every saved token accounted for. It runs inside the customer's own cloud, so nothing leaves their network, works with whatever provider and model they already use, and goes in within minutes. What you'll do Build your own pipeline. That means targeted outbound to engineering and platform leaders, starting in the Israeli ecosystem and moving into the US. Run deals start to finish: demo the product, handle the technical conversation yourself, and carry evaluations through to a signature. Own the GTM engine: the outbound, the deal process, the playbooks, the collateral. Get Anyray in front of the right people, on LinkedIn, at meetups and events, and through customers who'll vouch for us. Grow into leading the whole GTM function. The motion you build is the one you'll run. Who you are You know how to create your own momentum and don't wait to be handed a process. You've sold into the US market and are comfortable running those cycles remotely. You generate your own pipeline. Outbound is just how you work. You're technical enough to sell infrastructure without an SE in the room. You get LLMs, tokens and cloud deployment well enough to hold your own with a VP of Engineering. You want to build the function, not just hit a number, but you still love the close. Bonus points An existing network among engineering and platform leaders in Israeli tech. You've sold devtools, cloud, or AI infrastructure before. Why this role Full ownership of revenue and the motion behind it, a direct line to the founders, and a mandate that grows with everything you close. How to apply If that's you, email [email protected] and tell us about the hardest deal you've had to close.