uploaded files to Azure are corrupted when using dio

Question

I'm trying to upload a file from my phone to azure blob storage as a BlockBlob with a SAS. I can get the file to upload, but it can't be opened once downloaded. The file gets corrupted somehow. I thought this was a content-type problem, but I have tried several different approaches to changing to content-type. Nothing has worked so far.

My code:

FileInfo _fileInfo = await filePicker(); // get the file path and file name
// my getUploadInfo fires a call to my backend to get a SAS.
// I know for a fact that this works because my website uses this SAS to upload files perfectly fine
UploadInfo uploadInfo = await getUploadInfo(_fileInfo.fileName, _fileInfo.filePath); 

final bytes = File(_fileInfo.filePath).readAsBytesSync();

try {
  final response = await myDio.put(
    uploadInfo.url,
    data: bytes, 
    onSendProgress:
      (int sent, int total) {
        if (total != -1) {
          print((sent / total * 100).toStringAsFixed(0) + "%");
        }
      },
    options:
      dioPrefix.Options(headers: {
        'x-ms-blob-type': 'BlockBlob',
        'Content-Type': mime(_fileInfo.filePath),
      })
  );
} catch (e) {
  print(e);
}

This code uploads a file just fine. But I can't open the file since it becomes corrupted. At first, I thought this was a Content-Type problem, so I've tried changing the content type header to: application/octet-stream and multipart/form-data as well. That doesn't work.

I've also tried to do

dioPrefix.FormData formData =
  new dioPrefix.FormData.fromMap({
    'file': await MultipartFile.fromFile(
      _fileInfo.filePath,
      filename: _fileInfo.fileName,
    )
});
...
final response = await myDio.put(
    uploadInfo.url,
    data: formData, // This approach is recommended on the dio documentation
    onSendProgress:
...

but this also corrupts the file. It gets uploaded, but I can't open it.

I have been able to successfully upload a file with this code, but with this approach I cannot get any type of response so I have no idea whether it uploaded successfully or not (Also, I can't get the progress of the upload):

try {
  final data = imageFile.readAsBytesSync();
  final response = await http.put( // here, response is empty no matter what i try to print
    url, 
    body: data, 
    headers: {
      'x-ms-blob-type': 'BlockBlob',
      'Content-Type': mime(filePath),
  });
...

Any help would be greatly appreciated. Thanks

Solution

I tried to upload a file using dio in Dart to Azure Blob Storage, and then download and print the content of the file, as the code below.

import 'package:dio/dio.dart';
import 'dart:io';

main() async {
  var accountName = '<account name>';
  var containerName = '<container name>';
  var blobName = '<blob name>';
  var sasTokenContainerLevel = '<container level sas token copied from Azure Storage Explorer, such as `st=2019-12-31T07%3A17%3A31Z&se=2020-01-01T07%3A17%3A31Z&sp=racwdl&sv=2018-03-28&sr=c&sig=xxxxxxxxxxxxxxxxxxxxxxxxxx`';
  var url = 'https://$accountName.blob.core.windows.net/$containerName/$blobName?$sasTokenContainerLevel';
  var data = File(blobName).readAsBytesSync();
  var dio = Dio();
  try {
    final response = await dio.put(
      url,
      data: data, 
      onSendProgress:
      (int sent, int total) {
        if (total != -1) {
          print((sent / total * 100).toStringAsFixed(0) + "%");
        }
      },
      options: Options(
        headers: {
          'x-ms-blob-type': 'BlockBlob',
          'Content-Type': 'text/plain',
      })
    );
    print(response.data);
  } catch (e) {
    print(e);
  }
  Response response = await dio.get(url);
  print(response.data);
}

Then, I ran it and got the result as the figure below.

The content of the uploaded file as blob is the json string encoded from a Uint8List bytes from the funtion readAsBytesSync.

I researched the description and the source code of dio, actually I found dio is only suitable for sending the request body of json format, not for raw content as request body.

Fig 1. The default transformer apply for POST method

Fig 2. https://github.com/flutterchina/dio/blob/master/dio/lib/src/transformer.dart

So to fix it is to write a custom transformer class PutTransformerForRawData instead of the default one to override the function transformRequest, as the code below.

import 'dart:typed_data';

class PutTransformerForRawData extends DefaultTransformer {
  @override
  Future<String> transformRequest(RequestOptions options) async {
    if(options.data is Uint8List) {
      return new String.fromCharCodes(options.data);
    } else if(options.data is String) {
      return options.data;
    }
  }
}

And to replace the default transformer via the code below.

var dio = Dio();
dio.transformer = PutTransformerForRawData();

Then, you can get the data via the code below.

var data = File(blobName).readAsBytesSync();

Or

var data = File(blobName).readAsStringSync();

Note: the custom transfer PutTransformerForRawData is only for uploading, please remove the download & print code Response response = await dio.get(url); print(response.data);, the default transformer seems to check the response body whether be json format, I got the exception as below when my uploaded file is my sample code.

Unhandled exception:
DioError [DioErrorType.DEFAULT]: FormatException: Unexpected character (at character 1)
import 'dart:typed_data';

Comments

[Thiago Custodio]

try to download the file using Azure Storage Explorer. If it's also corrupted, then it's your upload code that is broken. PS: I've never heard something like that (SAS damaging the blob content. I'm 99.999% sure it's your download / upload code)

[wei]

It's also corrupted when using Azure Storage Explorer. Sorry, the title is misleading, I'll change it. I'm also 99.99999% sure it's the upload code. I'm just not sure what part of it is messing up

[Thiago Custodio]

Unfortunately there's no SDK for flutter. You'd better try to do it using REST gist.github.com/gregjhogan/ef37c38371193c8e9d08d867c05ad210 PS: use a proxy to capture the request and compare with your current one, you'll be able to identify what is different

[DCodes]

hi @wei , whenever I'm using this method im getting a 404 knowing that my url is correct and I'm unable to use a connection string with a SAS Token.. any help?

[DCodes]

hi @Peter I'm getting a 404 error when my url is written without a SAS Token ad a 403 error when my url is appended with a SAS Token.. any idea why? I'm sure my url is correct and the SAS is not expired.

[Mallikarjun Hampannavar]

@DCodes Did you solve the issue? I'm also getting same error when we add sas token to url 403 error

[DCodes]

@MallikarjunHampannavar check this out please :stackoverflow.com/questions/64196180/…

[Mallikarjun Hampannavar]

@DCodes thank you.. This I have checked and getting 201 as a status code.. Is it success ?

[DCodes]

yess @MallikarjunHampannavar. A 201 status code indicates that a request was successful and as a result, a resource has been created (for example a new page)..