FTP to Azure Blob Storage

Question

I had to setup secure FTP to Azure Blob Storage using popular FTP clients (like FileZilla, for example). After doing lot of research, I came across a link that says:

Deployed in a worker role, the code creates an FTP server that can accept connections from all popular FTP clients (like FileZilla, for example) for command and control of your blob storage account.

Following the instructions of the link, I had implemented the same and deployed the worker role on Azure production environment and it was successful. But still I am not able to connect the FTP host server (provided by me in configuration file) using FileZilla. I don't know what I had done wrong or missed anything.

Solution 1

But why?

There are already two very good FTP-style Azure Storage clients out there:
http://storageexplorer.com and http://azurestorageexplorer.codeplex.com

Both of them, as @Guarav well pointed out, can use a Shared Access Signature (SAS) to connect to Azure Storage without exposing the account key. You can then use a different SAS for each customer, if you're building a multi-tenant service - although if you think about it - that's not a very sound separation boundary.

I would use a separate storage account for every customer. That way if a storage account gets compromised, it only affects one customer. The following limit applies:

From https://azure.microsoft.com/en-us/documentation/articles/storage-scalability-targets/:

Scalability targets for blobs, queues, tables, and files

Number of storage accounts per subscription: 200

This includes both Standard and Premium storage accounts. If you require more than 200 storage accounts, make a request through Azure Support. The Azure Storage team will review your business case and may approve up to 250 storage accounts.

Solution 2

If you are okay with a little programming with Node.js, you can host a FTP server directly backed by Azure Blob.

You can use nodeftpd combined with azure-storage-fs. nodeftpd is the FTP server written in Node.js and support third-party file system manager. azure-storage-fs is a file system manager that is designed to use for nodeftpd and talks to Azure Blob directly.

The file system manager integration code is clearly written under README.md of azure-storage-fs. But you will need to write your own authentication code.

Solution 3

I came across this now (2022) and noticed that MS is now offering SFTP directly in Azure Storage. The feature is in Preview as of now though.

https://docs.microsoft.com/en-us/azure/storage/blobs/secure-file-transfer-protocol-support

Comments

[evilSnobu]

But.. why? There are already two very good FTP-style Azure Storage clients out there: storageexplorer.com and azurestorageexplorer.codeplex.com

[techV]

@evilSnobu thanks!!! ....so you mean to say I don't need to have ftp setup and i can do so by using azure storage explorer. also can upload and download the blob files.

[evilSnobu]

That's exactly right.

[techV]

yes.. i got it. i had spent almost two days figuring out the solution for this secure ftp connectivity. totally appreciated.. thanks buddy.

[techV]

@evilSnobu hey.. one thing what if we need to give access to our clients without telling them sensitive info like account key. as because in order to use storage explorer they need to have these info to connect.

[Gaurav Mantri]

Do take a look at storageexplorer.com. It lets you connect to your storage account using a Shared Access Signature which doesn't include the account key.

[techV]

@GauravMantri thanks... can you please also tell me where will i find or get the SAS URI... when i am trying to connect to individual blob it is asking me for SAS URI.

[Gaurav Mantri]

You would need to create a SAS URI either on a blob or the blob container (depending on what you're trying to do). You can create a SAS URI using this tool itself or programmatically. I would highly recommend reading azure.microsoft.com/en-in/documentation/articles/… to learn more about SAS. HTH.

[techV]

@GauravMantri great.. sure i will go through this. thanks for your time :)

[Gaurav Mantri]

@evilSnobu Would appreciate if you could put your comments as an answer.

[dreftymac]

See also: stackoverflow.com/questions/13195871/windows-azure-and-sftp

[JJ_Coder4Hire]

I do not consider the storage explorer to be a very good FTP-style transfer tool. I have not had much like with the retry/resume capability for large multi GB files.

[Eric Grover]

Often SFTP servers are used for vendors to send data feeds, and you can't just tell them to use Storage Explorer because the feed is automated from an existing system.

[evilSnobu]

In that scenario your best bet would be an FTP server in a VM (or Cloud Service) with an Azure File share as storage, so you can treat the compute part more or less as stateless. Service Fabric is also a good option with its Reliable Services programming model.

[evilSnobu]

fine, just spin up a Linux VM with vsftpd or something. OP never mentions automation once, however he does mention FileZilla twice.

[Sinaesthetic]

I like the idea, but it says that nodeftpd is not production ready

[sports]

I might be wrong but the storage explorer doesnt have all of the capacities of an ftp client such as: (1) download an entire container, (2) download all of the blobs from within a container (they are showed paginated so at most you could select all the blobs from one page and download only those)

[lightmotive]

Regarding sound separation boundary: recommend separate subscription for each client--most scalable, easy to automate, easy enough to manage, easy to transfer all client resources to a client owner if necessary (change directory, transfer billing ownership).

[piotr.gradzinski]

Just a heads-up - SFTP connection to Azure Storage Account is not working if the tool is using JSCH - docs.microsoft.com/en-us/answers/questions/713024/…