Revision as of 20:54, 11 May 2020

Storage settings allow storing files outside of your WHMCS installation, reducing web space requirements. Prior to version 7.7, WHMCS only supported local storage. With version 7.7, we have introduced Amazon AWS S3 and compatible services for file storage needs.

How it Works

By default, files are stored locally within the WHMCS installation. If you upgrade an existing WHMCS installation to version 7.7 or later, the system will retain any custom settings for the attachments directory or downloads directory. Any files will remain in their existing locations. However, you may opt to migrate select file types to Amazon AWS S3 or a compatible service. WHMCS offers automatic migrations of your existing files to and from AWS S3, as well as between AWS S3 locations.

Configuration

To manage your storage settings, navigate to Setup > Storage Settings. Choose one of the following workflows depending on the desired use case:

Local Storage

To migrate files from an existing local storage location to another local directory:

1. Go to Configuration () > System Settings > Storage Settings or, prior to WHMCS 8.0, Setup > Storage Settings.
2. Choose the Configuration tab.
3. Under Add New Configuration, choose Local Storage and click Add.
4. Enter path to the new storage location and click Save. The directory must exist and be writable.
5. Choose the Settings tab.
6. Select the new storage location for the file type that you wish to migrate to the new location.
7. Click Migrate when the system prompts you to migrate the files automatically.
   • For smaller scale deployments, migration will take less than a minute, and your files will then be stored at the new location.
   • If the migration cannot be completed immediately, it will be scheduled to run in the background. Until it is completed, the former location will continue to be used. Once all files are migrated to the new location, the setting will automatically switch to the new selection and begin using the new location.
   • Instead of using automated migration, you may opt to migrate your files manually and switch the relevant file type's storage setting immediately.
     • This will skip automatic migration and assumes your responsibility for moving the files to the new location.
     • Some file types may be stored by WHMCS in the same folder (for example, client files, email and ticket attachments as well as project management files will be stored in "attachments" folder). If you decide to move all the files away from that folder, all relevant file types must be switched to the new location.
8. After the automatic migration finishes, validate whether the migration was successful.
9. Delete the files in the prior location. The system will not automatically delete these files.

You may choose to cancel an ongoing migration. The present location will then continue to be used. Files migrated to the new location until this point will not be deleted.

If you make a change by mistake, click Revert Changes to cancel any changes.

AWS S3

After you configure the storage bucket, you may want to migrate files to an AWS S3 bucket or a compatible service. If you are familiar with AWS S3, follow these steps:

1. Click the "Configuration" tab.
2. In "Add New Configuration" tile, choose "S3" or and click "Add".
3. Enter AWS S3 connection details. You must specify the S3 region by its code name (for example, "us-east-1" for "US East (N. Virginia)").
   • To find the region code name for a bucket's region, click here.
   • The system doesn't require the "Endpoint URL" if you're using the Amazon AWS S3 service. This field is only required if you are using a non-Amazon, S3-compatible storage provider.
   • The S3 access credentials must have read and write access to the specified S3 bucket. See below for an example of a suitable policy for the S3 user.
4. Click "Save". If the system can't access the bucket, an error message will appear.
5. Click on the "Settings" tab.
6. Select the new storage location for the file type that you wish to migrate to the new location. Like using local storage, the system can migrate most file types automatically. The system will prompt you to start migration by clicking the "Migrate" button.
   • For smaller-scale deployments, migration will take less than a minute, and your files will then exist at the new location.
   • If the system can't complete the migration immediately, it will schedule it to run in the background. Until it finishes, the system will continue to use the former location. When the migration is finished, the setting will automatically switch to the new selection and begin using the new location.
   • The system won't automatically remove files from the former location. Do this manually after you verify that the migration was successful.

You may choose to cancel an ongoing migration. If you do, the system will continue to use the present location. Files that the system already migrated to the new location will remain.

Instead of using automated migration, you may opt to migrate your files manually and switch the relevant file types storage setting at once. This will skip automatic migration and you will be responsible for moving the files to the new location. If you use this method, note that WHMCS may store certain file types in the same folder (for example, the system will store client files, email and ticket attachments, and project management files in the "attachments" folder). If you decide to move all the files away from that local folder, you must switch all relevant file types to the new location.

If you made a change by mistake, click "Revert Changes" to cancel any changes.

Example Security Policy

Consider using the IAM Policy Generator tool in the AWS control panel, or the public policy generator tool. For power users, an example AWS S3 policy to assign to the S3 user for use with WHMCS storage settings is below:

   {
       "Version": "2012-10-17",
       "Statement": [
           {
               "Sid": "Stmt1111222223333",
               "Effect": "Allow",
               "Action": [
                   "s3:ListBucket",
                   "s3:GetObject",
                   "s3:GetObjectAcl",
                   "s3:PutObject",
                   "s3:PutObjectAcl",
                   "s3:ReplicateObject",
                   "s3:DeleteObject"
               ],
               "Resource": [
                   "arn:aws:s3:::<BUCKET_NAME>",
                   "arn:aws:s3:::<BUCKET_NAME>/*"
               ]
           }
       ]
   }

Note for manual migration only: The system stores files in AWS S3 with a prefix corresponding to the file asset type. For example, it will store file1.png, a client file, as /client_files/file1.png. Use the following sub-folder names per file type:

• Client Files — client_files
• Downloads — downloads
• Email Attachments — email_attachments
• Email Template Attachments — template_attachments
• Project Management Files — pm_files
• Ticket Attachments — ticket_attachments

When using automatic migration, you do not have to worry about these prefixes; the system manages them automatically.

Migrating Between Two AWS S3 locations

In addition to Amazon AWS S3, WHMCS supports AWS S3 compatible services like Ceph. However, WHMCS only supports automatic migration between two S3 locations if both locations use the same storage provider.

Migrating From S3 To Local Storage

You can perform this migration the same way as the other migration paths, with the same considerations and expectations equally applicable.

General Considerations

Note that the system can't automatically migrate "Email Attachments" (as opposed to email template attachments). You upload these files when creating a custom email message, and they are of a transient nature. To avoid any inconsistencies when switching storage settings for email attachments, it should be sufficient to ensure that no administrator users are in process of composing an email message with attachments. Note that email template attachments are a separate asset type that you can migrate automatically.

Troubleshooting a File Not Found Error

If you experience a "File not found" error, this may mean that a file is not present or is not accessible at the expected location. To validate that a file is present in the local storage, check that the file exists and its permissions allow access to the web user. For AWS S3, use AWS (or your compatible storage provider's) console to verify that the file is stored with the indicated key.

A convenient way to confirm storage location accessibility is to use the "Test Configuration" feature. To do that, go to Setup > Storage Settings, click "Configurations" tab, choose a configuration you want to test, and click the "Test" icon. For AWS S3 (or compatible providers) this should also indicate whether there are any issues with the provided Access Key / Access Secret.