How do I list all the paths and directories in my Firebase Storage with Flutter?

Question

I'm using Flutter and in my Firebase Storage I have a complex architecture of files and directories. I would like to list all the files contained in Firebase Storage (including their path) in order to download them and stock them on the local storage of the device.

I know how to list all the files from a folder, but how do I list the files from all the folders at the same time without knowing the name of these folders?

For instance, my architecture could look like this:

MainFolder
  -- Folder2
    -- Folder3
      -- file.json
    -- Folder4
      -- file2.json
AnotherMainFolder
  -- file3.json

The structure in my Firebase Storage is dynamic (I do not know the name and the number of directories I have inside it).

Until now, I have managed to writke the following code:

class DownloadFirebaseApi {
  static Future<List<FirebaseFile>> listAll() async {
    // here, I should specify a path `ref(path)`
    // however I don't want to because
    // I want all the folders and all the files
    // no matter in which directory they are
    final ref = FirebaseStorage.instance.ref();
    final result = await ref.listAll();
    final urls = await _getDownloadLinks(result.items);
    return urls
        .asMap()
        .map((index, url) {
          final ref = result.items[index];
          final name = ref.name;
          final file = FirebaseFile(name: name, url: url, ref: ref);
          return MapEntry(index, file);
        })
        .values
        .toList();
  }

  static Future<List<String>> _getDownloadLinks(List<Reference> refs) async {
    return Future.wait(refs.map((ref) {
      var url = ref.getDownloadURL();
      return ref.getDownloadURL();
    }).toList());
  }
}

Is there a recursive method that I don't know for listAll()?

Solution 1

When you call listAll the result has two lists:

1. result.items, which contains all individual files in the root.
2. result.prefixes, which contains all directories (commonly referred to as "prefixes" in Cloud Storage).

You'll want to loop over result.prefixes too, and call listAll on each of those in turn to get the full, recursive list of files.

Also see the FlutterFire documentation on listing files and directories.

Solution 2

For those who would be interested in knowing how I managed to code that (thanks to the answer of Frank van Puffelen)

import 'package:firebase_storage/firebase_storage.dart';

class FirebaseFile {
  final Reference ref;
  final String url;
  final String name;

  FirebaseFile({
    required this.ref,
    required this.url,
    required this.name,
  });

  @override
  String toString() {
    return 'file "$name"';
  }
}

class DownloadFirebaseApi {
  /// Transform the items (the files) contained in the [result] (a folder).
  /// The items are transformed into a list of [FirebaseFile] objects.
  static Future<List<FirebaseFile>> _getFilesFrom(ListResult result) async {
    List<String> urls = await _getDownloadLinks(result.items);
    return urls
        .asMap()
        .map((index, url) {
          final ref = result.items[index];
          final name = ref.name;
          final file = FirebaseFile(name: name, url: url, ref: ref);
          return MapEntry(index, file);
        })
        .values
        .toList();
  }

  /// Explores all the nested directories and
  /// returns a list of all the files as [FirebaseFile] objects.
  static Future<List<FirebaseFile>> _exploreDirectories(
    ListResult result,
  ) async {
    List<FirebaseFile> files = [];
    if (result.prefixes.isEmpty) {
      if (result.items.isEmpty) {
        return [];
      } else {
        return await _getFilesFrom(result);
      }
    } else {
      for (Reference prefix in result.prefixes) {
        ListResult nestedResult = await prefix.listAll();
        if (nestedResult.items.isNotEmpty) {
          files.addAll(await _getFilesFrom(nestedResult));
        }
        if (nestedResult.prefixes.isNotEmpty) {
          files.addAll(await _exploreDirectories(nestedResult));
        }
      }
    }
    return files;
  }

  /// Returns a list of all the files as [FirebaseFile] objects.
  static Future<List<FirebaseFile>> listAll() async {
    final ref = FirebaseStorage.instance.ref();
    List<FirebaseFile> files = [];
    ListResult result = await ref.listAll();
    if (result.prefixes.isNotEmpty) {
      files = await _exploreDirectories(result);
    } else if (result.items.isNotEmpty) {
      files = await _getFilesFrom(result);
    } else {
      return [];
    }

    return files;
  }

  /// Gets the download URL of every file with their [refs].
  static Future<List<String>> _getDownloadLinks(List<Reference> refs) async {
    return Future.wait(refs.map((ref) => ref.getDownloadURL()).toList());
  }
}