How to POST JSON Data to Remote API Using Coldfusion CFHTTP

Question

I'm sure that I'm completely botching this up but I got this far with the help of fellow Stack Overflow users, so thanks thus far.

I need to POST JSON data to a remote API. Obviously I can't use jQuery due to SOP issues, and the remote API does not support JSONP.

I also don't want to have to use any type of proxy as to get around the SOP limitations.

Per the API docs (http://myemma.com/api-docs/), this is the formatting of the data they expect (request and response data is transferred as JSON):

POST https://api.e2ma.net//123/members/add
{
  "fields": {
    "first_name": "myFirstName"
  }, 
  "email": "[email protected]"
}

And this is what I've built thus far but continue to receive "unable to parse JSON" errors from the remote API:

<cfset fields[name_first]="#SerializeJSON( "myFirstName" )#" />
<cfset form.email="#SerializeJSON( "[email protected]" )#" />

<cfhttp
  url="https://api.e2ma.net/123/members/add"
  method="POST"
  username="username"
  password="pssword"
  useragent="#CGI.http_user_agent#"
  result="objGet">

  <!--- add email --->
  <cfhttpparam
    type="formfield"
    name="email"
    value='#form.email#'
  />

  <!--- add field: name_first --->
  <cfhttpparam
    type="formfield"
    name="fields"
    value='#fields[name_first]#'
  />

</cfhttp>

<cfoutput>#objGet.FileContent#</cfoutput>

Again, I'm surely mangling the structure of my data somehow, but I'm not sure what I'm doing wrong, particularly regarding properly setting the "fields": { "first_name": "myFirstName" } structure/array.

Solution 1

You should send your request string as the httpparam type of body. The body of the request could be something like the entire form scope of your prepped structure. Be sure to either use array notation for setting your structure keys or put them in "quotes" during the implicit structure creation to ensure they retain their proper casing when the serializeJSON() takes place otherwise ColdFusion will uppercase the structure keys.

<cfset stFields = {
    "fields" = {
        "first_name" = "myFirstName"
     }, 
     "email" = "[email protected]"
}>   

<cfhttp url="http://api.url.com" method="post" result="httpResp" timeout="60">
    <cfhttpparam type="header" name="Content-Type" value="application/json" />
    <cfhttpparam type="body" value="#serializeJSON(stFields)#">
</cfhttp>

Update 10/26/13
For all the work I've been doing lately with APIs I thought I'd update an easy way to automate this casing that I've found. I've used a combination of the JSON Util library and Ben Nadel's JSON Serializer Utility CFC to accomplish much better serialization consistency for all returns.

Below is an example GIST of how I've implemented this.
https://gist.github.com/timmaybrown/7226809

As I've transitioned to using persistent entity CFCs in my projects, I've found that extending Ben Nadel's serializer CFC with my own child CFC method that loops all my persistent cfc's properties using the getComponentMetaData() function to build a structure of distinct keys and the casing for the serialization to follow. The approach allows my api to inherit automatically the casing of my property names within my entities and is very useful. A bit of overhead on reinit, but well worth it to keep your casing consistent in your API.

Update 9/8/16 Re: my point above about consistent casing. I have trended toward a different column naming convention in my databases for newer projects so I don't have to fight with a lot of these issues. first_name instead of firstName etc.

Solution 2

UPDATE: 9/26/2012: After requesting an API Key with the demo account I set up, they sent me one along with may account_id. I dropped the code in below and it worked like a charm for adding a member.

Let me start by saying that none of this code is tested (see update above). I don't have a MyEmma account, and apparently you have to be a paying customer for an account_id to use the API. That blows! But this should get you real close and may give you some ideas for encapsulating logic, which has become my obsession.

Secondly, I realize this post is 9 months old and you have probably either long figured it out, or won the lottery and are running the place by now. So no one may ever even see this post. But I was looking for some answers myself and ran across it... and since formulating and parsing JSON is part of my daily life, this is something I always need to keep setting myself straight on. So what turned out to be a quick answer to your question, became a late night, self serving, obsessive challenge. At any rate...

...what you are doing with JSON, is creating client side nested structures. You have the root structure with two key-value pairs (fields and email). Then the structure 'fields' holds a structure with the a key-value pair you are sending over for that email address (first_name). Presumably you can send more.

You are building nested structures. Remember that a key in a structure can hold a structure. And those keys can hold structures, and so on. It can get as dark and nasty as you want to go. But that's all JSON is... it's a client side object.

So here is your data build and JSON object...

<cfscript>
    variables.dataFields = {};
    variables.dataFields['fields'] = {};
    variables.dataFields['email'] = "[email protected]";
    variables.dataFields.fields['first_name'] = "myFirstName";
    variables.dataFields = serializejson(variables.dataFields);
</cfscript>

Note that I'm explicitly setting the structure key names with array notation. We have to do this to control the case with Coldfusion. Otherwise, the keys will be in all caps... not want we want for case sensitive JavaScript. This could be part of the problem you are having.

If Emma doesn't understand because of case, then you would get your...

{"error": "Unable to parse JSON request"}

But when we explicitly set our key names using array notation, and then serialize our object, we get nice and pretty, good ol' fashion JSON...

{"fields":{"first_name":"myFirstName"},"email":"[email protected]"}

So below, I put our http request to Emma in a function. It is also very important to set the Content-Type header as application/json, so the browser will send it as a object and not just a text string. And we are sending our JSON as the body of our request, not in a form field called 'fields'... hopefully that makes sense when you say it out loud. Here's the function...

<cffunction name="callEmma" access="private" displayname="CallEmma" description="This makes an HTTP REQUEST to MyEmma" returnformat="JSON" output="false" returntype="Any">
    <cfargument name="endpoint" required="true" type="string" displayname="EndPoint">
    <cfargument name="PUBLIC_API_KEY" required="true" type="string" displayname="PUBLIC_API_KEY">
    <cfargument name="PRIVATE_API_KEY" required="true" type="string" displayname="PRIVATE_API_KEY">
    <cfargument name="dataFields" required="true" type="struct" displayname="DataFields">
    <cfscript>
        local = {};
        local.baseURL = "https://api.e2ma.net/";
        local.account_id = "12345";
        local.phoneNumber = local.baseURL & local.account_id & arguments.endPoint;
        local.connection = new http();
        local.connection.setMethod("POST"); 
        local.connection.setUrl(local.phoneNumber);
        local.connection.setUsername(arguments.PUBLIC_API_KEY);
        local.connection.setPassword(arguments.PRIVATE_API_KEY);
        local.connection.setUserAgent(cgi.http_user_agent);
        local.connection.addParam(type="header",name="Content-Type", value="application/json");
        local.connection.addParam(type="body", value=arguments.dataFields); 
        local.objGet = local.connection.send().getPrefix();
        local.content = local.objGet.filecontent;
        return local.content
    </cfscript>
</cffunction>

Then once again, here is our JSON build (nested structures)...

<cfscript>
    variables.dataFields = {};
    variables.dataFields['fields'] = {};
    variables.dataFields['email'] = "[email protected]";
    variables.dataFields.fields['first_name'] = "myFirstName";
    variables.dataFields = serializejson(variables.dataFields);
</cfscript>

Then we set the variables to pass to the function...

<cfscript>
    variables.entryPoint = "/members/add";
    variables.PUBLIC_API_KEY= "PUBLIC_API_KEY";
    variables.PRIVATE_API_KEY= "PRIVATE_API_KEY";
</cfscript>

Then make the phone call...

<cfscript>
    variables.myResponse = callEmma(variables.entryPoint,variables.PUBLIC_API_KEY,variables.PRIVATE_API_KEY,variables.dataFields);
    variables.myResponse = deserializejson(variables.myResponse);
</cfscript>

We then take our response, deserialize it, and output the variables however we want.

<cfscript>
    if(variables.myResponse.added){
        writeoutput("Member " & variables.myResponse.member_id & " added!");
    }
    else{
        writeoutput("There was an error adding this member");
    }
</cfscript>

Anymore, I generally use <cfscript> as much as I can. It's easier to read and it makes me feel much smarter than I really am. So when we put it all together, for cut-and-paste, we have this...

<cfscript>
// Function to make our calls to Emma
private any function callEmma(required string endPoint,required string PUBLIC_API_KEY,required string PRIVATE_API_KEY,required string dataFields)
    description="This makes an HTTP REQUEST to MyEmma"
    displayname="CallEmma"
    returnformat="JSON"
    output="false"
{
    local = {};
    local.baseURL = "https://api.e2ma.net/";
    local.account_id = "12345";
    local.phoneNumber = local.baseURL & local.account_id & arguments.endPoint;
    local.connection = new http();
    local.connection.setMethod("POST"); 
    local.connection.setUrl(local.phoneNumber);
    local.connection.setUsername(arguments.PUBLIC_API_KEY);
    local.connection.setPassword(arguments.PRIVATE_API_KEY);
    local.connection.setUserAgent(cgi.http_user_agent);
    local.connection.addParam(type="header",name="Content-Type", value="application/json");
    local.connection.addParam(type="body",value=arguments.dataFields); 
    local.objGet = local.connection.send().getPrefix();
    local.content = local.objGet.filecontent;
    return local.content;
} 

// Put our data together
variables.dataFields = {};
variables.dataFields['fields'] = {};
variables.dataFields['email'] = "[email protected]";
variables.dataFields.fields['first_name'] = "myFirstName";
variables.dataFields = serializejson(variables.dataFields);

// Define the parameters for our call to Emma
variables.entryPoint = "/members/add";
variables.PUBLIC_API_KEY= "PUBLIC_API_KEY";
variables.PRIVATE_API_KEY= "PRIVATE_API_KEY";

// Call Emma
variables.myResponse = callEmma(variables.entryPoint,variables.PUBLIC_API_KEY,variables.PRIVATE_API_KEY,variables.dataFields);
variables.myResponse = deserializejson(variables.myResponse);

//Output to browser
if(variables.myResponse.added){
    writeoutput("Member " & variables.myResponse.member_id & " added!");
}
else{
    writeoutput("There was an error adding this member");
}
</cfscript>

My GOD! I've been writing WAY too many API's... I clearly need therapy!

Solution 3

The structure you have mentioned

{ "fields": { "first_name": "myFirstName" }, "email": "[email protected]" } In this JSON for 'fields' key value is again a JSON So, you can go like this:

<cfscript>
        VARIABLES.postJSON = StructNew();
        VARIABLES.nameJSON = StructNew();
        StructInsert(VARIABLES.nameJSON, 'first_name','myFirstName');
        StructInsert(VARIABLES.postJSON, 'fields',VARIABLES.nameJSON);
        StructInsert(VARIABLES.postJSON, 'email','[email protected]');
        
</cfscript> 

<cfhttp
  url="https://api.e2ma.net/123/members/add"
  method="POST"
  username="username"
  password="pssword"
  useragent="#CGI.http_user_agent#"
  result="objGet">

  <cfhttpparam
    type="body"
    name="field"
    value='#SerializeJSON(VARIABLES.postJSON)#'
  />

</cfhttp>

<cfoutput>#objGet.FileContent#</cfoutput>

Comments

[goxmedia]

I understand what you're saying, but I'm still getting the {"error": "Unable to parse JSON request"} back from the API. This means that I'm not submitting the "email": "[email protected]" or "fields": { "first_name": "myFirstName" } parameters correctly in my example above. That's what I need help with. Passing those JSON strings correctly.

[goxmedia]

You're describing a different issue but I appreciate the input.

[Dan Roberts]

If coldfusion presents any issues during serialization you can also just build up the body in a string such as <cfset stFields = ' "{ "fields" = { "first_name" = "#firstname#" }, "email" = "#email#" }' />

[timbrown]

yeah or fix some of the json Serialization issues in CF by optionally using the JSONUtil project link. has an option for strict mapping for key case sensitivity. Also you in some cases using javaCast('Boolean', 'true') will ensure gets set as a boolean rather than a string in the serialization. <cfset stFields = { "fields" = { "first_name" = "myFirstName", "is_active" = javaCast('Boolean', true) } }> This would result in this json string {"fields":{"first_name":"myFirstName","is_active":true},"ema‌​il":"[email protected]‌​m"}

[timbrown]

@user1113083 - if this was helpful you should mark this as the correct answer for others.

[Mark]

This is exactly what I needed for a call to an internal API that I didn't have access to the WSDL for. This worked perfectly. Thanks!

[timbrown]

@dadwithkids glad it helped you out. JSON serialization in CF is pretty straight forward, using quoted key names in your implicit array/struct creation and javaCast() on certain values as needed you can ensure the proper serialization quite easily.

[timbrown]

Updated answer with an additional approach to controlling serialization casing within Coldfusion...

[Nich]

@timbrown Would you kindly help here: stackoverflow.com/questions/23274469/…

[timbrown]

@nich - sure but I guess you figured it out and removed your question. Sorry I didn't see it earlier.

[timbrown]

@Gregory Matthews - if this question helped you solve your problem, please mark as the accepted answer to help others out.

[U.Malik]

@timbrown Perfect. Works for me :)