Firebase: How do I retrieve records from my data for which a specific key exists?

Question

I have data in firebase that looks like this:

"application": {
  "companies": {
    "firebase": {
      "creation": {
        "name": "Firebase Inc",
        "location": "USA"
      },

      "google": {
        "creattion": {
          "name": "Google Inc",
          "location": "USA"
        }
      }

      "facebook": {
      },

      "apple": {
      }
    }
  }
}

There are tens of thousands of records under companies key. How do i efficiently execute following queries?

How do I query only the records for which key creation is present under their name?

How do I query only the records that DO NOT have key creation present under their name?

I also want to call .on('child_added') on the returned result set so that I can process only those specific records later on. Is it possible?

EDIT: Simpler way without using an extra parameter

Queries

Here are the queries to do this without having to use an extra parameter:

• Find the companies without creation:
  • var ref = new Firebase(fbUrl+'/companies').orderByChild("creation").equalTo(null);
• Find the companies with creation:
  • var ref = new Firebase(fbUrl+'/companies').orderByChild("creation").startAt(!null);
• You would add ".indexOn": "creation" to the rules.

Edit 2: I was curious, so I pushed 11,000 records to /companies2 (half with creation children, half without). I was able to retrieve 5500 matching records in ~4 seconds using the above queries (or one of the variants I've shown below).

Edit 3: If you're running these queries frequently, it might be worth it to separate children of /companies into two bins based the presence of creation. That way, you can read the two segments separately without having to rely on queries.

Factory

Here is what the revised factory would look like (I've revised the PLNKR to match):

app.factory("CompaniesFactory",function($q, fbUrl){
  return function(hasCreation){
    var deferred = $q.defer();
    var ref = new Firebase(fbUrl+'/companies').orderByChild("creation");
    var query;
    if (hasCreation) {
      query = ref.startAt(!null);
      // or: 
      // query = ref.startAt(true);
    } else {
      query = ref.equalTo(null);
      // or:
      // query = ref.endAt(!null);
      // query = ref.endAt(true);
    }
    query.once("value", function(dataSnapshot){
      deferred.resolve(dataSnapshot.val());
    }, function (error) {
      deferred.reject(error);
    });
    return deferred.promise;
  }
});

And yes, it is possible to call .on('child_added') on the returned dataSnapshot. See DataSnapshot.ref().

---

Original answer using an extra parameter:

(Keeping this for reference)

Another way to do it would be by adding another parameter called hasCreation to children of companies that have creation, and query by that.

Data

• The query would then be var ref = new Firebase(fbUrl+'/companies').orderByChild("hasCreation").equalTo(hasCreation);
  • If hasCreation in the query is null, the query will return the companies without a hasCreation child.
  • If hasCreation in the query is true, the query will return the companies with hasCreation===true.

{
  "company1" : {
    "creation" : {
      "name" : "company1"
    },
    "hasCreation" : true
  },
  "company2" : {
    "name" : "company2"
  },
  "company3" : {
    "name" : "company3"
  },
  "company4" : {
    "creation" : {
      "name" : "company4"
    },
    "hasCreation" : true
  }
}

Rules

You would add the ".indexOn" : "hasCreation" to your rules like so:

  "so:29179389":{
    ".read" : true,
    ".write" : true,
    "companies" : {
      ".indexOn" : "hasCreation"
    }
  }

Companies Factory

app.factory("CompaniesFactory",function($q, fbUrl){
  return function(hasCreation){
    var deferred = $q.defer();
    if (!hasCreation) {
      hasCreation = null;
    }
    var ref = new Firebase(fbUrl+'/companies').orderByChild("hasCreation").equalTo(hasCreation);
    ref.once("value", function(dataSnapshot){
      deferred.resolve(dataSnapshot.val());
    });
    return deferred.promise;
  }
});

Controller

app.controller('HomeController',function($scope,fbUrl,CompaniesFactory) {
 $scope.getCompanies = function(hasCreation) {
  var companies = new CompaniesFactory(hasCreation).then(function(data){
     console.log(data);
     $scope.companies = data;
   });
 }
});

HTML

<body ng-app="sampleApp">
  <div ng-controller="HomeController">
    <button ng-click="getCompanies(true)">Find with creation</button>
    <button ng-click="getCompanies(false)">Find without creation</button>
    <h2>Companies:</h2>
    {{companies}}
  </div>
</body>

Comments

[Waseem]

The problem with this approach, that I'm also facing is, that my companies array is very large. Iterating over tens of thousands of records to get lets say only five records that have creation key in them brings my application to a halt.

[Waseem]

I added the information of data set being large in the questions.

[sbolel]

@kato/@frank, if you see this, are any of the variants from my last edit preferred as best practice? !null doesn't really make sense, but it works the same as true. Would .endAt have an advantage over .equalTo?