Rename != Replace


bug

 

This is a little post about how the recent change from Visitors to Users and Visits to Sessions affected me.   On April 16, 2014 the Google Analytics development team followed behind the Google Analytics website team in renaming a bunch of Dimensions and Metrics in Google Analytics Core Reporting API.   This change didn’t go smoothly for me.

What was the change?

Basically all the change did was create new columns and deprecate the old ones.

  •  All visitor and visit based dimensions and metrics have been renamed to instead use user and session respectively.

The change log also noted that deprecation would follow the normal guidelines of deprecated columns.

  • The following dimensions and metrics have been renamed. Review the Data Deprecation Policy for details on data renaming:

Renaming policy.

Data Rename
When columns are renamed:

  • The changelog of the corresponding API will be updated with details about the original and replacement column(s).
  • In the Metadata API , the status attribute of the original column will be set to DEPRECATED and the replaced By attribute will be set to the Id of the replacement column.
  • The original column(s) will continue to work in reporting queries for a minimum of 6 months.
  • After 6 months, the original column(s) will be removed from the Metadata API and querying for the column(s) in the reporting API will result in a 400 Bad Request error.

I wasn’t worried

I have email notification setup whenever anything is added to the change log I know about it.  When I saw that they were going to be following the normal depreciation policy I wasn’t worried, I have 6 months to deal with any changes that may affect my application.     I am the lead developer on TARGIT for Google Analytics so when something goes wrong with the application I’m one of the first to know about it.   Unfortunately this change hit right on Easter vacation so I wasn’t in the office when the emails started coming in.

Unfortunately it took me several hours to track down the problem.

The Problem

I am going to be chopping up the JSon results a little here you don’t need the full JSon response to see the problem.

https://www.googleapis.com/analytics/v3/data/ga?ids=ga:&dimensions=ga:visitorType&metrics=ga:visits,ga:bounces&start-date=2013-11-01&end-date=2013-12-01

Here you have a normal request you would send to Google Analytics Core Reporting API to request data the only thing it is missing is  &oauth_token= and a valid access token.

As you know the data returned by the API looks something like this, or it should.

 "columnHeaders": [
  {
   "name": "ga:visitorType",
   "columnType": "DIMENSION",
   "dataType": "STRING"
  },
  {
   "name": "ga:visits",
   "columnType": "METRIC",
   "dataType": "INTEGER"
  },
  {
   "name": "ga:bounces",
   "columnType": "METRIC",
   "dataType": "INTEGER"
  }
 ],
 "totalsForAllResults": {
  "ga:visits": "27715",
  "ga:bounces": "16276"
 },
 "rows": [
  [
   "New Visitor",
   "17034",
   "11998"
  ],
  [
   "Returning Visitor",
   "10681",
   "4278"
  ]
 ]

In order to correctly map the output columns my application uses the columnHeaders to match up each item in the rows.    I suspect that it isn’t needed I’m pretty sure that the API will always return them in the order I requested them, but pretty sure is never enough for me when dealing with an application and customers any errors might affect.   So I use the columnHeaders returned by the API to ensure that I am mapping values correctly.      My application knows that it requested ga:visitorType and now it knows that ga:visitorType is the first item in the row array returned by the API.

What the API returned

"columnHeaders": [
  {
   "name": "ga:userType",
   "columnType": "DIMENSION",
   "dataType": "STRING"
  },
  {
   "name": "ga:visits",
   "columnType": "METRIC",
   "dataType": "INTEGER"
  },
  {
   "name": "ga:bounces",
   "columnType": "METRIC",
   "dataType": "INTEGER"
  }
 ],
 "totalsForAllResults": {
  "ga:visits": "27715",
  "ga:bounces": "16276"
 },
 "rows": [
  [
   "New Visitor",
   "17034",
   "11998"
  ],
  [
   "Returning Visitor",
   "10681",
   "4278"
  ]
 ]

Do you see the problem? The API returned a columnHeader of ga:userType not ga:visitorType which is what I requested. So being unable to find the correct output column to match with the column being returned by the API my application just did nothing with the data. Which in turn gave me nulls in the database and cased issues with processing the SSAS cube.

IMO this is a major issue, if I request one column I should get that column back. Lets say I run the following select from the database.   Select Make from Car ,  what would happen if i suddenly got back the color of the car?  What does that do to your data integrity? Technically speaking ga:visitorType is ga:userType  isn’t it?   Not really one is a deprecated other isn’t they are different in more than just name, and if I request one I should get it back not the other.

Bug Report

I reported the bug immediately, and found that it was also in the Meta-Data and Real-time APIs, I also saw a report that said the v2 API was totally broken I can’t confirm this as i didn’t bother testing with v2 only v3. Request returning incorrect header name. Being that I am a Google Developer Expert for Google Analytics I have contact directly with the API team. I emailed them right away asking if they could fix it. Unfortunately I had to  patch my application in order to fix the problem, i couldn’t leave customers with a non functioning application.

Its Fixed!

Whether or not this bug affected you I am happy to report that on June 4, 2014 they released a fix to the API Release 2014-06-04 (June 4, 2014). I know for a fact that it has actually been fixed for a while now, I helped to test it.

Conclusion

I think that it is wonderful that the Google Analytics API team is constantly developing and improving the API, its one of the reasons I enjoy working with it.      I also applaud them on addressing the issue promptly, while it took them 3 weeks to get a working version up they did look into it immediately.      Just remember when you release a bug accidentally into production,  not even Google is perfect but its how you react to the issue that counts.      As programmers we cant be 100% sure that upgrades wont effect our users,  but we can be 100% sure that we deal it promptly if it does.


About Linda Lawton

My name is Linda Lawton I have more than 20 years experience working as an application developer and a database expert. I have also been working with Google APIs since 2012 and I have been contributing to the Google .Net client library since 2013. In 2013 I became a a Google Developer Experts for Google Analytics.

Leave a comment

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.