Dynamically creating a multi-dimensional hash in Ruby

Question

I'm a PHP developer who's trying to gain some proficiency in Ruby. One of the projects I'm cutting my teeth on now is a source-code auditing tool that scans webapp files for potentially dangerous functions in several web programming languages. When matches are found, the script saves the relevant information in a poi (point-of-interest) class for display later on.

An example instance of that class would look something like this (modeled in YAML):

poi:
    file_type: "php"
    file: "the-scanned-file.php"
    line_number: 100
    match: "eval()"
    snippet: "echo eval()"

On display, I want to organize these points of interest like so:

- file_type
-- file
--- match (the searched payload)

Thus, before presentation, I'm trying to structure a flat array of poi objects into a hash mirroring the structure above. This will allow me to simply iterate over the items in the hash to produce the desired on-screen organization. (Or at least, that's the plan.)

And now, for my question: how do I do that in Ruby?

In PHP, I could do something like this really easily:

<?php

$sorted_pois = array();
foreach($points_of_interest as $point){
    $sorted_pois[$point->file_type][$point->file][$point->match][] = $point;
}

?>

I've tried translating that thought from PHP to Ruby like this, but to no avail:

sorted_pois = {}
@points_of_interest.each_with_index do |point, index|
    sorted_pois[point.file_type.to_sym][point.file.to_sym][point.match.to_sym].push point
end

I've spent a few hours on this, and I'm kind of banging my head against the wall at this point, so presumably I'm way off-base. What's the proper way to handle this in Ruby?

Update:

For reference, this is the precise method I have defined:

# sort the points of interest into a structured hash
def sort
  sorted_pois = {}
  @points_of_interest.each_with_index do |point, index|
    sorted_pois[point.file_type.to_sym][point.file.to_sym][point.match.to_sym].push point
  end
end

This is the error I receive when I run the code:

./lib/models/vulnscanner.rb:63:in `sort': undefined method `[]' for nil:NilClass (NoMethodError)
    from /usr/lib/ruby/1.8/rubygems/custom_require.rb:31:in `each_with_index'
    from ./lib/models/vulnscanner.rb:62:in `each'
    from ./lib/models/vulnscanner.rb:62:in `each_with_index'
    from ./lib/models/vulnscanner.rb:62:in `sort'
    from ./webapp-vulnscan:69

Line 62 (as you can likely infer) is this line in particular:

@points_of_interest.each_with_index do |point, index|

As an additional reference, here's what (a snippet of) @points_of_interest looks like when converted to YAML:

- !ruby/object:PoI 
  file: models/couponkimoffer.php
  file_type: php
  group: :dangerous_functions
  line_number: "472"
  match: `
  snippet: ORDER BY `created_at` DESC
- !ruby/object:PoI 
  file: models/couponkimoffer.php
  file_type: php
  group: :dangerous_functions
  line_number: "818"
  match: `
  snippet: WHERE `company_slug` = '$company_slug'
- !ruby/object:PoI 
  file: models/couponkimoffer.php
  file_type: php
  group: :dangerous_functions
  line_number: "819"
  match: `
  snippet: ORDER BY `created_at` DESC

Solution 1

@John's Enumerable#group_by suggestion is one good way to solve your needs. Another would be to create an auto-vivifying Hash (like you appear to have in PHP) like so:

hash = Hash.new{ |h,k| h[k] = Hash.new(&h.default_proc) }
hash[:a][:b][:c] = 42
p hash
#=> {:a=>{:b=>{:c=>42}}}

Note that this sort of auto-vivification can be 'dangerous' if you access keys that don't exist, as it creates them for you:

p hash["does this exist?"]
#=> {}

p hash
#=> {:a=>{:b=>{:c=>42}}, "does this exist?"=>{}}

You can still use the vivifying default_proc without hitting this danger if you use key? to test for the key first:

val = hash["OH NOES"] if hash.key?("OH NOES")
#=> nil

p hash
#=> {:a=>{:b=>{:c=>42}}, "does this exist?"=>{}}

---

FWIW, the error you are getting says, "Hey, you put [] after something that evaluated to nil, and nil doesn't have a [] method." Specifically, your code...

sorted_pois[point.file_type.to_sym]

evaluated to nil (because the hash did not yet have a value for this key) and then you attempted to ask for

nil[point.file.to_sym]

Solution 2

You might be interested in group_by.

Sample usage:

birds = ["Golden Eagle", "Gyrfalcon", "American Robin",
         "Mountain BlueBird", "Mountain-Hawk Eagle"]
grouped_by_first_letter = birds.group_by { |s| s[0] }

# { "G"=>["Golden Eagle", "Gyrfalcon"], "A"=>["American Robin"],
#   "M"=>["Mountain BlueBird", "Mountain-Hawk Eagle"] }

Solution 3

The obvious problem with the example above is that nested hashes and arrays you try to use don't exist. Try this:

sorted_pois = {}
pois.each do |point|
  # sanitize data - convert to hash of symbolized keys and values
  poi = Hash[ %w{file_type file match}.map do |key| 
    [key.to_sym, point.send(key).to_sym]
  end ]

  # create nested hash/array if it doesn't already exist
  sorted_pois[ poi[:file_type] ] ||= {}
  sorted_pois[ poi[:file_type] ][ poi[:file] ] ||= {}
  sorted_pois[ poi[:file_type] ][ poi[:file] ][ poi[:match] ] ||= []

  sorted_pois[ poi[:file_type] ][ poi[:file] ][ poi[:match] ] << point
end

Comments

[Phrogz]

+1 for being right; you may gather more upvotes if you show how it's used beyond linking to the docs.

[Phrogz]

This is the 'safer' way to manually create the nestings; see my answer for a less-safe-but-more-convenient way.

[Chris Allen Lane]

@Phrogz, thanks for taking the time to explain that to me. I'm really starting to like Ruby, but man, it's tricky! This makes it obvious I've got a bit more reading to do :)

[BlackHatSamurai]

Great answer! Really helped me understand! Thank you for explaining things as well as you did.