Story ·
Railway’s security disclosure: what to rotate, and why
Railway is a platform developers use to run their apps. Overnight on 21–22 October it said an unauthorised party may have read secrets that customers store with it, including API keys and database passwords. Here is what Railway says, what it has not said yet, and what to do.
The short version
- What happened: Railway says it detected unusual activity in the systems that store customers’ environment variables, the settings where apps keep their secrets.
- What may be exposed: API keys, database passwords and private registry credentials. Railway says an unauthorised party “may have” read them, and for some, got the decrypted contents.
- Who is affected: not yet clear. Railway’s first look suggests “a subset of users”, but it asks everyone to act.
- What to do: change every secret stored on Railway, starting at the service that issued it. Do this at the provider first. You do not need Railway for that step.
- Still unknown: how the attacker got in, and how many accounts were affected.
First, what Railway holds for you
When you run an app on a platform like Railway, the app needs secrets: the key that lets it use an AI service, the password to its database, the token that lets it pull code. Those are stored on the platform as environment variables, and Railway keeps them encrypted.
A secret that has leaked works for whoever holds it until it is cancelled. That is why the advice is to change the secret itself, not just to watch for trouble.
How it unfolded
- Detection. Railway says it “detected anomalous activity” in systems that store customer environment variables. It gives no exact time.
- Disclosure. Railway publishes its notice and pauses all deployments, so customers cannot roll out new versions until builds are restored. It tells customers to roll every environment variable and private registry credential, and to review logs.
- Guides. Railway publishes step-by-step guides for rotating Postgres, MySQL, MongoDB and Redis credentials, and for checking for unexpected use of OpenAI, Anthropic, Grok, OpenRouter, Stripe, AWS, GCP and GitHub. It also says it reset all storage bucket credentials and the GitHub tokens it held, and that customers must reconnect GitHub themselves.
- Status. Railway’s status page still shows a major outage: builds are not back, and the next update is due by 8:30 am IST.
Why AI keys
Railway says it believes this is “part of a series of attacks recently targeting platform companies with the aim of compromising AI inference providers”, and that it has told those providers to revoke keys in advance. It does not name the attackers.
In plain terms, an AI provider’s API key lets whoever holds it use the owner’s account, and the owner pays the bill. A leaked key is money, which would explain why they are the target. That is our explanation, not Railway’s statement, and Railway has not said what the attacker actually did.
What to do
If you run anything on Railway
- Start at the provider, not at Railway. Cancel and replace each AI key, database password, cloud key and payment key at the service that issued it. This works even while Railway builds are down.
- Check what the old secrets did. Review usage and billing at each provider for activity you do not recognise. Railway’s guides list what to look for in OpenAI, Anthropic, AWS and others.
- Store the new secrets, then redeploy when builds return. Changing a variable usually needs a redeploy, which has to wait. Services that use Railway’s reference variables pick up a new database password on their next deploy. A pasted connection string does not.
- Reconnect GitHub. Railway reset the GitHub tokens it held and removed registry tokens from services. Those registry tokens still work on GitHub until you revoke them yourself.
- Use your audit log. Railway’s resets appear there as “Railway Staff”, so you can tell them from your own team’s changes.
If you only use apps that run on Railway
- Nothing to install. It is up to each app’s owner to change their secrets. If an app you use tells you something, follow its advice.
What we don’t know yet
- How the attacker got in. Railway has not given a cause.
- How many accounts are affected. Railway says it will email the ones it confirms.
- Whether the keys that decrypt the secrets were also taken. Railway says only that some were decrypted.
- When builds will resume.
This story will be corrected if Railway’s account changes. We have not seen any public evidence of misuse of specific customer secrets.
