Setting up Phoenix Framework and Ecto to use UUIDs: how to insert the generated value?

Question

A few days ago I started using Elixir and the Phoenix Framework (v 0.12.0) with a Postgres database. I'm trying to create a table which has a UUID primary key, which I prefer over the sequential default.

After using mix phoenix.gen.html to generate the model and migration files and following the other steps in the Phoenix docs, I have changed

def model do
    quote do
      use Ecto.Model
    end
  end

in web.ex to

def model do
  quote do
    use Ecto.Model
    @primary_key {:id, :uuid, []}
    @foreign_key_type :uuid
  end
end

as is mentioned in the Ecto docs. I have also changed the migration to

create table(:tblname, primary_key: false) do
  add :id, :uuid, primary_key: true
  [other columns]
end

Unfortunately, when I try to add an entry to the table from the auto-generated form, I get an error because the id is null. If I manually add an id-column to the model, I receive an error that the column already exists. If I neglect to set primary_key to false in table/2 and remove the id column, the table is generated with a sequential id-column.

Do I need to manually set the id in the changeset, or have I made an error in setting up my app to use UUIDs? Thanks in advance

Solution 1

EDIT: I have updated this answer to Ecto v2.0. You can read the previous answer at the end.

Ecto v2

Handling UUIDs in Ecto has become much more straight-forward since the original answer. Ecto has two types of IDs: :id and :binary_id. The first is an integer ID as we know from databases, the second is database specific binary. For Postgres, it is a UUID.

To have UUID as primary keys, first specify them in your migration:

create table(:posts, primary_key: false) do
  add :id, :binary_id, primary_key: true
end

Then in your model module (outside the schema block):

@primary_key {:id, :binary_id, autogenerate: true}

When you specify the :autogenerate option for :binary_id, Ecto will guarantee that either the adapter or the database will generate it for you. However, you can still generate it manually if you prefer. Btw, you could have used :uuid in your migration and Ecto.UUID in your schema instead of :binary_id, the benefit of :binary_id is that it is portable across databases.

Ecto v1

You need to tell your database how to automatically generate the UUID for you. Or you need to generate one from the application side. It depends which one you prefer.

Before we move on, it is important to say that you are using :uuid that will return binaries instead of a human readable UUIDs. It is very likely you want to use Ecto.UUID which will format it as a string (aaaa-bbb-ccc-...) and that's what I'll use below.

Generating in the database

In your migration, define a default for the field:

add :id, :uuid, primary_key: true, default: fragment("uuid_generate_v4()")

I am assuming you are running on PostgreSQL. You need to install the uuid-ossp extension with CREATE EXTENSION "uuid-ossp" in pgAdmin or add execute "CREATE EXTENSION \"uuid-ossp\"" in the migration. More information about the UUID generator can be found here.

Back to Ecto, in your model, ask Ecto to read the field from the database after insert/update:

@primary_key {:id, Ecto.UUID, read_after_writes: true}

Now, when you insert, the database will generate a default value and Ecto will read it back.

Generating in the application

You will need to define a module that inserts the UUID for you:

defmodule MyApp.UUID do
  def put_uuid(changeset) do
    Ecto.Changeset.put_change(changeset, :id, Ecto.UUID.generate())
  end
end

And use it as a callback:

def model do
  quote do
    use Ecto.Model
    @primary_key {:id, Ecto.UUID, []}
    @foreign_key_type Ecto.UUID
    before_insert MyApp.UUID, :put_uuid, []
  end
end

before_insert is a callback and it will call the given module at the given function with the given arguments, with a changeset representing what is being inserted being given as first argument.

That should be all. Btw, there is a chance this will be more streamlined in the future. :)

Solution 2

Also when creating a new project pass option --binary-id to use UUID as default primary key.(Starting Ecto v2)

mix phx.new project_name --binary-id

Comments

[Theemuts]

Thank you for your quick reply! Unfortunately, both approaches lead to an error in Postgrex. If I generate the UUID in my app, I receive the following error: no function clause matching in Postgrex.Extensions.Binary.encode/4 I received the same error if I chain Ecto.Changeset.put_new_change/3 to the pipeline in the model to add the UUID to the parameters. If I let Postgres generate the UUID by calling uuid_generate_v4(), Postgrex complains the function does not exist: (Postgrex.Error) ERROR (undefined_function): function uuid_generate_v4() does not exist

[Theemuts]

After executing CREATE EXTENSION "uuid-ossp" in pgAdmin uuid_generate_v4() is working, the row is created correctly. The index view returns a UnicodeConversionError, however.

[José Valim]

I am not sure why Postgres is complaining about uuid_generate_v4(). Maybe you need to install some module? I don't remember the steps exactly so I have added a link to the docs. The other bugs are because you are using :uuid in the model instead of Ecto.UUID, I have explained why you want the later in the second paragraph.

[José Valim]

I have also updated the code samples so others won't do the same mistake as us! Thanks for the feedback.

[Josh W Lewis]

CREATE EXTENSION "uuid-ossp" needs to be ran by the same database user that Ecto is using. So if Ecto is connecting as 'ecto-user', run the command as 'ecto-user'. Creating extensions doesn't seem to span accounts.

[Josh W Lewis]

I take that back, CREATE EXTENSION "uuid-ossp"; is database specific. Make sure you are connected to the database Ecto is using before installing the extension. I had to install it on both my dev and test databases.

[José Valim]

You can also do it in a migration! execute "CREATE EXTENSION \"uuid-ossp\""

[websymphony]

Since ecto '0.12.1' using uuids for primary keys has become more streamlined. Here is an example gist for setting it up: gist.github.com/websymphony/48dbfc8e663da8a0d63d

[tumbudu]

Another import thing is specifying reference type as uuid. add :post_id , references(:posts, type: :uuid)

[Qqwy]

I believe callbacks are deprecated in newer versions of Ecto. How do we generate default values (that cannot be known at compile-time) in the application now?

[Paul Fioravanti]

This answer and the Ecto.Schema docs helped me get UUIDs working for Postgres.

[Trevoke]

Tried the solution for v2 like this: Mud.Repo.insert(%Mud.Room{description: "Room", exits: %{"w" => 1}}) getting an error that id is null and is not allowed to be null.

[Trevoke]

Never mind, got it - found another blog entry that showed me what I had misunderstood. Suggested edit to this response for clarification.