AWS EB and RDS: Invalid storage size for engine name mysql and storage type gp2

Question

I'm trying to deploy my (python/django) application to AWS Elastic Beanstalk, but I am getting the error. Deployments have been working fine for the last few months, so I'm confused what could be the cause. Here's the error:

ERROR: Updating RDS database named: aa1xxxxxxxxx failed Reason: Invalid storage size for engine name mysql and storage type gp2: 5

Someone helped me with the initial EB+RDS setup, so I'm not 100% familiar with how all of it was setup from the get go. However, now when I go into the AWS console, I see a RDS instance that has a storage field set to 100GiB and when I go into the EB configurations, I see a connection to my existing RDS instance, but the storage here only shows 5 GB. I tried updating the 5GB to 10GB, but I still get the same error and deployment still doesn't work.

Also, the Endpoint in my EB database settings looks to be the same as the url for my RDS instance's endpoint `aa1xxxx.yyyyyyyyy.us-west-2.rds.amazonaws.com.

AWS RDS settings

AWS Elastic Beanstalk Database Configs

Note: I found this thread, which was similar, but not exactly my situation since I was able to edit the storage field, even though that didn't work.

Solution

It looks like the RDS instance has been modified outside of Elastic Beanstalk - which means that the configuration data between what EB believes to be true, and what is actually true, no longer match up. This is a common problem when using tools like Elastic Beanstalk - it's just a wrapper for other AWS services (it's actually a nice UI for a very specific set of CloudFormation templates), and there's nothing stopping you going to the RDS console and fiddling with the settings yourself once EB has provisioned them.

gp2 has a minimum storage size of 20GB for MySQL instances, so the reason you're getting this error is because your EB environment's 5GB storage size configuration conflicts with the storage type set on the RDS instance. EB believes the storage type to be magentic, so won't enforce the limitations set out by the other storage types. It's not Elastic Beanstalk emitting that error, it's the CloudFormation stack sitting behind EB that ties everything together that the EB console ultimately controls.

In order to resolve this, you'll want to set your Elastic Beanstalk's environment to match up as close as you can to what your RDS instance is configured to. I don't believe it's possible with Elastic Beanstalk to modify the RDS instance's storage type from the default magnetic configuration. So, you've got 2 options here:

1. Set your storage size in your EB config to be 100GB and save it, this should have EB effectively sync up with the truth. When it applies, it'll tell CloudFormation to resize the database. Now whether this will actually take the instance down for resizing or not, since the "new" storage size is the same as the current size, I'm not sure. It may also throw an error because new size == old size. You could modify it to the minimum 20GB (if you're wanting to go way lower), which will do an instance resize, and ensure that your storage size on RDS matches that of Elastic Beanstalk's configuration. You'll just have to bear in mind that you can't go any lower than 20GB because of the storage type limitation.

2. Resize your RDS instance manually back to magentic storage and size of 5GB - to what Elastic Beanstalk believes to be the truth. This of course will only work if your database is small enough to fit on a 5GB instance. If it doesn't, you're out of luck here, and it's probably worth exploring option #1 instead. If it will fit, this action will resize your database back to what Elastic Beanstalk thinks it currently it, and you'll then be able to resize your database again using EB's configuration tools.

Comments

[Deven]

ok, i'll try this out. but i did want to clarify what will happen if "this will actually take the instance down for resizing or not"? if the instance goes down, is all the data on my database gone? do i need to do some backups before trying to set storage in my EB console to 100GB? Thanks in advance for all your help thus far!

[Deven]

amazing--setting my EB config for the database to 100GB fixed the problem and my deployment worked like a charm! thank you for the great answer!!

[dannosaur]

@Deven when an instance is taken down for resizing, RDS's internals will provision a new instance (or volume, unsure tbh) and copy all your data across to it, bring it online then shut the old instance/volume down. You never lose data during this process.