I've been tasked with getting our AWS environment cleaned up, and I'm pretty sure we have some resources that are no longer being used. I need something that can look for potential cleanup targets, but I definitely don't want anything deleted until I've explicitly approved it. Preferably I should be able to approve the suggested changes before the cleanup runs.

If you wired a Bedrock agent action group to an existing Lambda by hand (CLI, Terraform, CDK or SAM), read that function's resource-based policy once more.

One shape that is easy to end up with is one statement letting the service principal bedrock.amazonaws.com call lambda:InvokeFunction, with no Condition block. A plain aws lambda add-permission --principal bedrock.amazonaws.com gives exactly that; the Lambda docs say to add --source-arn to restrict invocations to a specific resource, but nothing forces you to.

IAM's confused deputy page says such a policy authorises the service principal, "and not the actor who configured the calling service". So the statement no longer says which agent, or which account, may cause the call. AWS's own Bedrock example adds AWS:SourceAccount (your account ID) and AWS:SourceArn (arn:aws:bedrock:<region>:123456789012:agent/<agent-id>), and calls both optional and recommended, which makes them easy to skip.

Scoping the agent service role's trust policy does not close this: same principal, separate document, different question. Nor does checking agent.id from the input event inside the handler. That is useful for logging which agent called, but the invoke was already allowed by then.

To check: run aws lambda get-policy --function-name <name> on every action group function. Any statement with "Service": "bedrock.amazonaws.com" and no Condition is the one to review. Compare its SourceArn with the agents that should call it.

Trade-off: pinning aws:SourceArn to one agent ID means each new or re-created agent (new ID) needs a policy update, or its calls fail. aws:SourceAccount alone is simpler and may be fine in a dedicated account where agent creation is tightly controlled, at the cost of letting any agent there reach the tool. And put-resource-policy replaces the existing policy, so read the current one first.

Docs: https://docs.aws.amazon.com/bedrock/latest/userguide/agents-permissions.html (resource-based policy section) and https://docs.aws.amazon.com/IAM/latest/UserGuide/confused-deputy.html

For those running Bedrock agents in shared accounts: do you pin SourceArn per agent ID and update the policy on each new agent, or stop at SourceAccount and control who can create agents? Has anyone enforced this centrally (a CI policy check or a resource control policy) rather than per function?

Hey r/aws,

Some of you might remember Kosty, I posted about v1.8 here a while back (YAML filtering, tags/ARNs, the React dashboard). You gave me a lot of good feedback and honestly a good chunk of what follows came out of those comments, so thanks.

Quick catch-up: it's at 3.6.1 now. Since 1.8 I added a GenAI audit for Bedrock and SageMaker, rebuilt the whole thing around plugins in 3.0, and went a lot deeper on security. It's at 151 security and cost checks across 30 services now. Still free, still MIT, still just pip install kosty.

Why I rewrote it as plugins

By 1.8 the code had grown in every direction and adding a single service meant touching the core in three or four places. It was getting painful even for me. Since 3.0 every service is its own plugin with its own checks and the CLI just picks them up. If you want to add a check you don't need to understand the whole codebase anymore, which I hope means more people will actually contribute.

Security

This is where most of the work went.

• Attack surface mapping. It looks at 15 resource types that can end up public (ALB, EC2, S3, RDS, API Gateway, Lambda URLs, CloudFront, OpenSearch, EKS, ECR, snapshots…) and tells you for each one whether it's unprotected, partially protected or actually protected. Mostly so you know where to start on Monday morning.
• IAM privilege escalation. It checks for 21 known escalation patterns. Pattern-matching policies gives you false positives, so there's a --deep flag that confirms each finding with SimulatePrincipalPolicy.
• Org-wide scans. Runs in parallel across all your accounts with cross-account role assumption. Works fine on a few hundred accounts.
• The usual hardening stuff. WAF (managed rules, rate limiting, bot control, logging, count vs block), API Gateway (auth, throttling, TLS 1.2, CloudFront bypass, request validation), plus CloudTrail, GuardDuty, VPC Flow Logs, Config and KMS rotation.

​

kosty public-exposure --output console
kosty iam check-privilege-escalation --deep
kosty audit --organization --max-workers 20 --output all

GenAI (Bedrock / SageMaker)

This is the part I'd really like feedback on. A lot of teams spun up Bedrock and SageMaker pretty fast over the last year and from what I see almost nobody audits it yet.

• Bedrock, 12 checks: missing guardrails, shadow AI (models being used that nobody approved), VPC endpoints, prompt caching you're not using, inference profiles, custom model encryption, invocation logging, budget limits, TPM quotas, cross-account model access, model sizing, and stuff that could run as batch.
• SageMaker, 8 checks: idle endpoints (the classic GPU quietly burning money all weekend), zombie notebooks, Spot training, checkpointing, Inference Components, VPC endpoints, internet access, root access.

​

kosty ai audit
kosty ai bedrock check-shadow-ai
kosty ai sagemaker check-idle-endpoints

What didn't change

Cost findings still come with actual dollar amounts (11 services), multi-profile YAML config and tag/ARN filtering are still there, and the dashboard now has a separate AI/ML view.

What's missing

No diff between two scans yet, and no PDF report for management. Both are on the list, just not done.

If you want to try it:

pip install kosty
kosty audit --output all

Repo: https://github.com/kosty-cloud/kosty

Two things I'd love to hear about:

1. If you run Bedrock or SageMaker in prod, what misconfig actually bit you? I'd rather build the next checks from real stories than from my own guesses.
2. Is --deep worth the extra API calls for you, or should it just be on by default?

Sharing here for more public visibility on poor AWS Support service. If anyone from AWS happens to see this, the case number is `179232184700097`.

1. Friday - Open request quota increase for EC2 Spot Instances from the AWS Console.
   
2. Monday - Request still not handled. Open chat and get told it was routed to another dept.
   
3. Monday - Get a confirmation that CloudFront was enabled (nothing to do with my request).

It's totally clear that AWS Support has degraded substantially in recent years. I'm definitely a "power user" of AWS and I'm not asking for or expecting any solutions support for $50 / month on the account.

However, a quota increase is such a straightforward ask and included details of the use case and why it was important. There was absolutely no way that someone could mistake this for CloudFront.

I'm sure the AWS Support team have a lot on. We all do! The basic stuff should be handled better!

I had an intrusion on one of my servers, which I then fixed. i changed my root password, checked my CloudTrail logs, I have 2FA active.

I received an email saying "Thank you for completing the required remediation steps to secure your AWS account.

I see that our service team placed restrictions on your account because of this security event. I’ve asked them to remove the restrictions. etc..."

Now suddenly I get an email saying that my account has been suspended. Email says "Your AWS account may have been accessed by an unauthorized third-party. We contacted you about this issue and requested that you take action to secure your account. Because we did not receive a response by the requested date, we have suspended your account."

But this is not true, I took all the steps and recieved an email saying everything was fine.

Now all my production sites are down!

I have a case open : 179249730300928 if anyone from AWS is here.

I’m a project owner, not a database engineer. I’ve been using ChatGPT for guidance/review and Codex for implementation to build an independent, encrypted backup system for my Supabase database using AWS.

The goal is to recover our application data, Auth records, database roles, permissions and RLS policies reliably. Storage files and full managed-service recovery are separate concerns; we haven’t established those yet.

An encrypted backup exists in AWS, and verification/decryption works. The part we still cannot complete is an actual restoration of that backup.

Our setup uses AWS S3/KMS and CodeBuild, with restoration into an isolated, disposable database using a pinned Supabase/Linux PostgreSQL 17.6 runtime. These tests do not restore over production.

We’ve gone through several rounds of failures and corrections:

• Some early blockers were defects in our own retention checks and role-comparison code.
• Those were corrected, and role restoration now passes its internal comparisons.
• Subsequent attempts fail during database content restoration.
• Our diagnostic code initially removed so much detail to protect private information that we couldn’t identify the underlying error. Several rounds of work have gone into correcting that.

Latest actual restoration result:

• Archive verification and decryption passed.
• Role/bootstrap restoration and internal role/grantor comparisons passed.
• Database bootstrap and extension setup passed.
• pg_restore failed during content restoration with exit status 1.
• The retained diagnostic points near TOC entry 4046 — EVENT TRIGGER, but this is only a candidate location.
• We still don’t have the actual SQLSTATE or a confirmed failing statement.
• Final content/security comparisons and recovery probes weren’t reached.
• Cleanup completed, and production and the original backup remain unchanged.

One potential lead is the event-trigger ownership handling in Supautils v3.4.0. However, we haven’t established that those checks caused this failure or even applied to the failed session.

The latest local change adds correlated PostgreSQL JSON server logs so we can obtain the real SQLSTATE and source location while keeping private SQL and data inside the disposable environment. That change hasn’t been tested in the hosted restoration yet.

We now have 903 passing local tests, plus 14 fresh native PostgreSQL checkpoints for the latest diagnostic work. But the native tests use stock PostgreSQL on Windows, which clearly doesn’t prove that our actual backup restores on the Supabase/Linux runtime.

Honestly, I’m frustrated. I keep getting reports that tests passed, followed by another failed restore and another diagnostic change. I’m relying on the AI tools for technical decisions, and I’m concerned we’ve built too much custom machinery around something that should have a more established approach.

For people who have successfully implemented and tested independent Supabase backups:

1. Have you encountered event-trigger or Supautils ownership issues when restoring into a disposable Supabase/Postgres instance?
2. What exact evidence should we collect next to identify the failure without exposing user data?
3. What backup/restore approach do you use to preserve application data, Auth records and security behavior?
4. Does this sound like a restore-environment or implementation problem, and what would you simplify first?

I’m looking for an experienced sanity check and a practical path to a demonstrably working restore. I understand nobody can diagnose the exact cause without the actual error.

I'm trying to determine whether it would be worth the effort to host a chat app backend on AWS Lambda now that Lambda Web Functions (https://aws.amazon.com/blogs/aws/announcing-aws-lambda-web-functions-for-fast-consistent-web-app-performance-anywhere/) came out. I'll be using Postgres RDS for the database and I don't want to have to think of managing servers ideally.

Haven't started writing the backend for it yet but I was thinking of choosing either Node.js or Go depending on if it'd be worth trying out web functions. The chat would need to be in real-time but WebSockets don't seem to be supported yet so making use of SSE could work. Budget wise I'd like to keep the costs as low as possible, ideally under $50/mo.

Would it be worth building out the app's backend on Lambda or should I just go with an EC2 instance for now?

I have a few domains with AWS, 4 in play at the moment and I've been trying to buy another.

No matter what I try and buy I see

"We are not sure about the availability of jhdshjjhdskj.com
You can proceed with registration but it may fail if the domain is not available."

I've checked my card and its fine, it even billed this month to AWS without issue.

The error message is not very helpful and basically says put a support case in but as I'm on a basic plan I'm not expecting any priority.

We recently received an online request from your AWS account to register the following domain name: somedomain.com We weren't able to register the domain name. This happened for the following reason(s): We can't finish registering your domain. Contact AWS Support at https://console.aws.amazon.com/support/home?region=us-east-1#/case/create?issueType=customer-service&serviceCode=service-domains&categoryCode=registration-issue  for further information.

Has anyone seen this before and knows of a fix I can try myself ?

Thanks,

Update

This is the response from AWS

Your account is currently restricted from performing additional domain registrations. Unfortunately, we are unable to lift this restriction at this time. We apologize for the inconvenience.

No further info seems no way to appeal? I don’t know what I did wrong? I’ve been with AWS for 10 + years

Update 10/20/2026

On why this happened: This restriction comes from a service-level adjustment that AWS applies to domain registrations for a set of accounts. It is not triggered by anything you did, and it does not reflect a judgment about your account, your payment history, or your tenure as a customer. It is a constraint tied specifically to the Route 53 domain registration service, not to your overall account. On what you can do: Unfortunately, there is no action you can take to lift this restriction. It is applied and managed on the AWS side, and we are unable to override it. We understand that is a frustrating answer, but we want to be straightforward with you rather than give you steps that would not lead anywhere.On your broader concern about S3 and other services: This restriction is specific to new domain registrations through Route 53. It does not affect your ability to use S3, EC2, or any other AWS service, and it does not indicate that broader restrictions are being applied to your account. Your account is in good standing.

Maybe naively I thought AWS was beyond this silliness but for an engineering company to say I've done nothing wrong and it just is is disappointing.

Hi everyone,

We have critical applications running on both Amazon Aurora MySQL (v3/8.0 compatible) and Amazon Aurora PostgreSQL. And to protect our core applications from rogue queries or resource contention, our current policy forces all ad-hoc data queries, analytics, and basic reporting users to connect exclusively to the Cluster Reader Endpoint.

However, our team needs to frequently perform live performance analysis and triage active database bottlenecks (such as tracking query resource consumption, analyzing transaction lock contention, and generating execution plans).

Coming from an Oracle background where the global data dictionary views (like GV$ and DBA_HIST) shows statistics across all cluster instances instantly. However, we are running into node-isolation hurdles currently in these databases. For example, standard tools like pg_stat_activity or performance_schema tables appear to only expose data local to the physical node we are connected to.

My Questions:

1. If our team is logged strictly into an Aurora Reader Node, is there any native catalog extension or view (e.g., aurora_stat_activity or replica_host_status) that allows us to safely inspect the activity like waits events, thread states, locks, or processlists running on the Writer Node? Or are these metadata views strictly localized to the instance processing the connection?

2. What are the best-practice strategies for giving terminal access to a primary Writer node for triage of issues without risking accidental high-cost data scans that could impact the primary application traffic on write node?

3. For deep-dive profiling—specifically running an EXPLAIN or EXPLAIN ANALYZE on an active operational statement—is it physically required to open a direct terminal connection to the specific Writer Instance Endpoint to capture accurate planner behavior?

I need TTS but compliance dictates AWS... and it seems Polly is the best they have to offer which... well... leaves a lot to be desired.

Why doesn't AWS just host one of the better open-source TTS and charge their markup?

Saw this asked last year, wanted to give it another crack now that AI has had a chance to grow. Everyone feel free to chime in your experience choosing and why, especially if it diverges from the kind of expected "our needs aren't super compex so kube is overkill". I'm asking primarily for myself though, and my situation is as follows:

I run a small startup (seven figure revenue/profit, single digit staff) and have been wanting to make our infra more legit.

We have some light to moderate jank (too much stuff in the same subnet, public (empty whitelist) subnets that should be private, etc. - also I kinda hate CloudFormation), and have been floating a cleanup/rewrite, and I find myself wondering if we should just go ahead and do EKS.

We're currently on ECS with two services and don't expect to ever have more than a dozen (and there's a limit to how big we can get in terms of tapping out our niche, and I don't want a ton of services floating around with a small crew).

We will probably never go multicloud.

And now my questions:

The general idea seems to be that EKS is overkill and easier to mess up, but with AI getting better, leaning on it is a stronger option. I don't want to make things harder for myself but EKS likely has a much bigger corpus than ECS and may actually end up being easier in many workflows. Also I'm a dev and have gotten my hands quite dirty kubeing my way around enterprise sandbox and dev environents (before AI). Unless I'm missing something, I think a simple EKS is really not that bad. Anything missing or misguided in this thinking?

It is important to me to have our stuff as super-enterprise-standard as possible, to do right by my employees (mostly juniors) and set them up for bright futures in the industry when they choose to move on. I know ECS is used a lot, but EKS feels much, much stronger for this - does this hold true?

If you run AI agents, LLM apps or MCP servers in your AWS environment, you've probably been asked a simple question that's hard to answer: where is this agent actually sending data?

Once an agent can call models and tools for itself, the traffic gets hard to follow. Platform and security teams usually want three things. They want to see every flow. They want to stop or hold certain calls before they go out. And they want a record of what happened, without sending all of that to yet another outside service.

ControlTower is an open-source, self-hosted AI gateway for LLM and MCP traffic, built for that problem. You run it yourself. It gives you a live map of every agentic data flow, and you can gate, approve, inspect and account for each call that passes through it.

In the console you can see every flow, trace a single agent, draw a gate on a path, and approve a call that's being held. It's released under the Apache-2.0 license.

Repo: https://github.com/joshmaster2165/controltower

A question for people already running agents on AWS: how do you handle this today? Do you route LLM and MCP calls through a central gateway or proxy, rely on logging after the fact, or leave it to each team? And where would a human approval step actually help, versus just get in the way?

Disclosure: I'm posting this on behalf of the github.com/joshmaster2165/controltower team.

A lot of small product teams end up with the same setup on AWS. Resources were created in the console during the early rush, so there's little or no infrastructure as code. There's a Kubernetes cluster nobody fully owns, a CI/CD pipeline that breaks in ways only one person understands, and a bill that keeps creeping up. Fixing it always loses out to feature work.

The usual options are to hire, to have the product engineers fix it on the side, or to bring in consultants for a set period. Each one has tradeoffs, and the consultant route seems to go well or badly depending on scope and handoff.

For context, I'm posting for RaynX.ai, an engineering services company that works on this kind of problem. Their cloud and infrastructure work covers Terraform/IaC, Kubernetes, cloud migration and cost optimization. They also do DevOps and CI/CD, API architecture (gateway setup, OAuth/OIDC, service mesh) and platform modernization. On the application side they build SaaS platforms, AI and ML systems (RAG pipelines, model integration, vector search), data and analytics infrastructure, and MVPs. Their past work includes enterprise API gateways on MuleSoft Flex, Mule Gateway and Cloudflare, plus a customer analytics platform.

They also make VidForgeX TMA, which may interest ops-minded teams. It turns screen recordings and meetings into searchable records, time and motion breakdowns, workflow audits and SOP compliance checks. It works inside Claude and Cursor through MCP.

More details are here: https://www.raynxai.com

I'd like to hear from people who have been through this. If you brought in outside help for IaC, Kubernetes or pipeline work, what made it worth it or not? Was it clear scope, the documentation left behind, or how well your team could run things afterward? And if you kept it in-house, what made that the better call?

Disclosure: I'm posting this on behalf of the raynxai.com team.

Hey everyone,

I'm a 2025 BCA graduate, and I've recently received an offer for an Assistant Technical Support Engineer role at ChargePoint. The role involves night shifts from 8:30 PM to 5:30 AM, and I've decided to accept it to gain corporate exposure, build practical skills, and get some industry experience.

I'm still figuring out whether this is the right career move, but my current plan is to work on my technical skills alongside my job, particularly in computer networking, cloud infrastructure, and AWS certifications. In the long term, I’m also considering pursuing an MS in a field that aligns with my career goals.

I'd like to know whether this is a realistic plan if I'm consistent and willing to put in the work.

I'm particularly interested in computer networking and would appreciate some guidance on the following:

- Is computer networking a good career path in terms of job opportunities, career growth, and salary?

- How should a beginner start learning networking, and what skills or certifications should I focus on first?

- Would learning networking alongside AWS and cloud infrastructure be a good combination?

- What are the chances of transitioning into a networking or cloud-related role at a good company after gaining some experience?

- For those who have pursued a similar path, what would you recommend doing differently?

I'd really appreciate advice from experienced professionals, especially networking engineers and people working in cloud infrastructure.

I'm willing to put in the effort and build my skills, but I want to make informed decisions about my career rather than blindly pursuing certifications.

Thanks in advance for your guidance!

I have AWS credits and want to use them for GPU inference. Which GPU instances and services do the credits actually cover? From your experience, what are the right steps to get GPU access (quota, region, setup) without hitting walls? Any traps that bill outside the credits?

On October 24, AWS is running a pretty significant test of its European Sovereign Cloud.

For a few hours, it'll operate without a connection to the AWS global network backbone. Traffic will be routed through European internet providers instead, and the system used for limited operational data transfers outside the EU will be disabled.

AWS expects services to remain available, although there might be brief connectivity interruptions when the routes switch.

It's one thing to design a cloud to operate independently. Actually testing it under these conditions is another.

This could be a big step for cloud sovereignty in Europe, especially for organisations dealing with strict regulatory requirements.

The results should be worth paying attention to.

Hi, for a few weeks I'm receiving emails with "AWS Budgets: Account xyz has a budget exceeding your alert threshold" but I don't recognise the account. It might be a very old account of mine that I forgot to delete, but the only one I have access to is not linked to the account. AWS attempted to charge my credit card but luckily it's a prepaid and didn't have enough funds, so I blocked it just to be sure.

Contacted support with my known account, they told me they couldn't do anything and I should create another case with account recovery while signed out, but I'm getting no response: case 179198505900161.

Is there another way to contact support and escalate to resolve this issue? The amount luckily wasn't big, but i would like to resolve the issue, close any old open account and avoid the possibility of debt collection due to an account that is potentially compromised and I do not have access anymore. My other options is to try with a registered letter to AWS EMEA asking for GDPR data, at least they would then legally have 30 days to respond...

Any other suggestion?

Hey AWS,

Earn some trust, and be customer obsessed and help me with a support case that gets no attention.

I am trying to sign up for an account and input my payment card details and get

"Error 880104: Sorry, there was an error processing your request. Please refresh the page and try again. If the error persists, contact AWS Support "

Support case ID: 179200018700806

The best part is the same exact credit card is on another AWS account and gets AWS charges just fine.

I have tried 3 other cards, no virtual cards, private browsers, etc. Nothing works.

I tried Q, I tried the automated instructions in the support case, all of which either provide instructions that cannot be followed because I dont have a complete account with payment method or dont exist. This is wild AWS that I have to reach out on reddit to get you to respond.

Please help, my project is waiting.

Hi, so my server is in maintenance mode, so we are just keeping it running till we can. Some stuff is running on AL1 and I guess it would be made private by Oct 30th and I guess I need to create an AMI such that the process is running fine before we migrate.

I am a total noob and might be sounding stupid, can some AWS experienced person show me the light here, whether what I am thinking is correct or we can do something better. Since, I have only like 13 days. Thanks !

What's happens on AWS US East 2? Loading login page takes so long.

Even loading 200KB of a CSS file takes 22 seconds

Hi everyone,

I'm an individual developer trying to use OpenAI GPT-5.6 Sol through Amazon Bedrock.

The model is visible in the Bedrock Playground, but when I send a simple message, I immediately receive this error:

"ValidationException: Access to OpenAI models is not allowed from unsupported countries, regions, or territories."

I've checked my AWS account information and selected a supported AWS Region, but I'm still unable to invoke the model.

I also have approximately $100 in AWS new-account credits that I was hoping to use for Bedrock API experimentation.

I'm trying to understand:

1. Does Bedrock enforce OpenAI's geographical restrictions based on the caller's public IP, AWS tax address, billing country, or a combination of these?

2. Can this error occur even when the AWS Region and account billing information appear valid?

3. Are standard AWS Free Tier promotional credits eligible to cover proprietary OpenAI model inference on Bedrock, or only certain Bedrock models?

4. Could using the Bedrock Runtime endpoint with cross-Region inference introduce additional account-plan restrictions?

I would appreciate any documentation or firsthand experiences that could help identify the actual cause before I make further configuration changes.

Thanks!