UPDATE: HOWTO Mailchimp's OAuth2

I have been working to integrate the Mailchimp OAuth implementation into an application that uses mailchimp mailing lists to drive call campaigns. My users login into my site using mailchimp, so you can see its going to be a very strong integration.

I have found a few problems with the implementation. This is about my 10th OAuth implementation so I am pretty familiar with all the little implementation differences. Here are a few crits, meant to be constructive. 

Here are some details. I am using the Mailchimp python API on bitbucket and the Bananas_OAuth implementation.

Mailchimp Python API - https://bitbucket.org/mailchimp/mailchimp-api-python
Banana Py Oauth - https://github.com/claytantor/Banana-Py/blob/master/banana_py/__init__.py

1. Its seems as though the calls from https://login.mailchimp.com/oauth2/metadata don't return enough information to reliably create a user in the authenticating system and that the API does not have a method to get data on the OAuth user that the access token was generated for. This creates a problem when attempting to create a user for the first time. When user authenticates the following information should be in the response: email address, unique unchangeable id, and mail chimp username. Right now I have to use the "account_name" which can be changed. Can you see where this might be an issue? I also have to generate a fake email and force the user to verify the email when I could just use the one you verified already. This can be dealt with using the account-details method in the helper api.
2. The OAuth implementation does not implement scope.
3. The OAuth implementation does not use best practices for refresh tokens.
4. The OAuth implementation is difficult to implement a flow for, I have implemented about 10 of these and I couldnt figure it out from your docs. If Bananas wast there I would have ended up spending an insane of of time.
5. Making the API docs say that the calls use the API key is confusing I had to read deep to see that you could use an access token to make calls.
6. Why doesn't it have a OAuth implementation in the API, c'mom just add Bananas_OAuth and do a little more architectural work to unify the concepts.

UPDATE: Cole from Mailchimp API got back to me in a day and gave me a solution for #1, I added this to the Gist as well. Looks like it works.

I have a Gist that shows some of the backflips I am having to do, its too bad too because with some very small changes the OAuth2 implementation and API would be pretty darn good. Check out what I am doing:

https://gist.github.com/claytantor/1d1169e0cc6d77d2c3ff

	# AUTH
	# ******************************************************************************************
	# Step 1: Your application begins the authorization process by redirecting the user to the authorize_uri
	#
	# - this is a GET request
	# - response_type=code, your client_id, and the *urlencoded* redirect_uri are included
	# ******************************************************************************************
	#
	# authorize_uri = https://login.mailchimp.com/oauth2/authorize?response_type=code&client_id=635959587059&redirect_uri=http%3A%2F%2F192.168.1.8%2Foauth%2Fcomplete.php
	def auth_mc(request):
	bannanas = Bananas_OAuth()
	return redirect(bannanas.authorize_url())
	# ******************************************************************************************
	# Step 4: Your application must make an out-of-band request to the access_token_uri using the "code" returned
	#
	# - This is a POST request
	# - as you can see, grant_type, client_id, client_secret, code, and redirect_uri are *all* POSTed
	# ******************************************************************************************
	#
	# access_token_uri: https://login.mailchimp.com/oauth2/token
	#
	# REQUEST:
	#
	# POST /oauth2/token HTTP/1.1
	# User-Agent: oauth2-draft-v10
	# Host: login.mailchimp.com
	# Accept: application/json
	# Content-Length: 198
	# Content-Type: application/x-www-form-urlencoded
	#
	# grant_type=authorization_code&client_id=635959587059&client_secret=0da3e7744949e1406b7b250051ee1a95&code=1edf2589e664fd317f6a7ff5f97b42f7&redirect_uri=http%3A%2F%2F192.168.1.8%2Foauth%2Fcomplete.php
	def redirect_mc(request, tempate_name='redirect_mc.html'):
	bannanas = Bananas_OAuth()
	#{'access_token': 'secret', 'scope': None, 'expires_in': 0}
	bannanas_auth = bannanas.authenticate(request.GET['code'])
	# what the user info that comes from bananas
	# This is a bad approach because what we really need here
	# is some user info so we can either look up the user if they already exist
	# or create the user if it doesnt. Using the accountname is not a good idea
	# because it can change.
	# {
	# "login_url": "https://login.mailchimp.com",
	# "access_token": "secret",
	# "expires_in": 0,
	# "dc": "us1",
	# "accountname": "Your Account Name Can Change Inc.,",
	# "api_endpoint": "https://us1.api.mailchimp.com",
	# "role": "owner",
	# "scope": null
	# }
	if bannanas_auth['access_token']:
	#try to get the user info
	#account-details
	mc = mailchimp.Mailchimp(bannanas_auth['access_token'])
	#account-details(string apikey, array exclude)
	details = mc.helper.account_details()
	try:
	cp_user = CallpugUser.objects.get(username=details['user_id'])
	cp_user.access_token = bannanas_auth['access_token']
	cp_user.save()
	except ObjectDoesNotExist:
	#use the mailchimp user id which will not change
	cp_user = CallpugUser.objects.create_user(
	details['user_id'], details['contact']['email'],
	settings.CALLPUG_SECRET_KEY)
	cp_user.integration_type='mailchimp'
	cp_user.integration_id=details['user_id']
	cp_user.access_token=bannanas_auth['access_token']
	cp_user.save()
	# authenticate the user, this shouldnt use the account name because it can change
	# we use the mailchimp username_i to authenticate
	print 'authenticating: {0}:{1}'.format(dedtails['user_id'],settings.CALLPUG_SECRET_KEY)
	auth_user = authenticate(username=details['user_id'],
	password=settings.CALLPUG_SECRET_KEY)
	if auth_user is not None:
	if auth_user.is_active:
	login(request, auth_user)
	# Redirect to a success page.
	#needs the full app url for redirect
	return redirect(reverse('user_home'))
	else:
	# # Return a 'disabled account' error message
	# context['message']=request.POST['username']+' account has been suspended.'
	return render_to_response('error.html',{'message':'auth user is not empty but us unactive'},
	context_instance=RequestContext(request))
	#flail
	return render_to_response('error.html',{'message':'unknown problem with login'},
	context_instance=RequestContext(request))

view raw views.py hosted with ❤ by GitHub

I am also giving you a reference to an OAuth implementation that is solid and easy to implement:

https://developer.github.com/v3/oauth/

Now you tell me.

Play 2.0 Support is Now Part of the Scala Plugin

The key change we’d like to highlight is that the Scala and Play 2.0 plugins have merged together, so with IntelliJ IDEA 14 you don’t need to install them separately because Play 2.0 support is now a part of the Scala plugin. You still need to run IntelliJ IDEA Ultimate to enable Play 2.0 support though.

Why Google's Oauth2 implementation is poor

When comparing oauth2 providers I find that Google, while being the most compliant has the absolute worst working examples of their code. Everytime I see example code from google I get the mental image picture of a 24 year old developer who claims "its works!" while not even considering how someone who has never used the technology they are proposing may interpret their completely uncommented code base.

	import os
	import logging
	import httplib2
	from apiclient.discovery import build
	from django.contrib.auth.decorators import login_required
	from django.core.urlresolvers import reverse
	from django.http import HttpResponse
	from django.http import HttpResponseBadRequest
	from django.http import HttpResponseRedirect
	from django.shortcuts import render_to_response
	from django_sample.plus.models import CredentialsModel
	from django_sample import settings
	from oauth2client import xsrfutil
	from oauth2client.client import flow_from_clientsecrets
	from oauth2client.django_orm import Storage
	# CLIENT_SECRETS, name of a file containing the OAuth 2.0 information for this
	# application, including client_id and client_secret, which are found
	# on the API Access tab on the Google APIs
	# Console <http://code.google.com/apis/console>
	CLIENT_SECRETS = os.path.join(os.path.dirname(__file__), '..', 'client_secrets.json')
	FLOW = flow_from_clientsecrets(
	CLIENT_SECRETS,
	scope='https://www.googleapis.com/auth/plus.me',
	redirect_uri='http://localhost:8000/oauth2callback')
	@login_required
	def index(request):
	storage = Storage(CredentialsModel, 'id', request.user, 'credential')
	credential = storage.get()
	if credential is None or credential.invalid == True:
	FLOW.params['state'] = xsrfutil.generate_token(settings.SECRET_KEY,
	request.user)
	authorize_url = FLOW.step1_get_authorize_url()
	return HttpResponseRedirect(authorize_url)
	else:
	http = httplib2.Http()
	http = credential.authorize(http)
	service = build("plus", "v1", http=http)
	activities = service.activities()
	activitylist = activities.list(collection='public',
	userId='me').execute()
	logging.info(activitylist)
	return render_to_response('plus/welcome.html', {
	'activitylist': activitylist,
	})
	@login_required
	def auth_return(request):
	if not xsrfutil.validate_token(settings.SECRET_KEY, request.REQUEST['state'],
	request.user):
	return HttpResponseBadRequest()
	credential = FLOW.step2_exchange(request.REQUEST)
	storage = Storage(CredentialsModel, 'id', request.user, 'credential')
	storage.put(credential)
	return HttpResponseRedirect("/")

view raw gistfile1.txt hosted with ❤ by GitHub

Really? So I attempted to get this framework working in my app for hours. It requires that you use the google model objects and the xsrfutil libs to work (I guess I never got it to).

https://developers.google.com/accounts/docs/OAuth2WebServer

and

https://code.google.com/p/google-api-python-client/source/browse/samples/django_sample/plus/views.py

This FLOW framework obscures the oauth2 process from the developer, so you are completely dependent on the code base being simple to understand to get the implementation working. Secondly creating the oauth2 app client credentials are buried deep in google's "API Dashboard". It was the longest amount of time I have ever spent on an oauth2 implementation. I have done many of these and this one is very difficult to work with.

Compare this to oauth2 providers that actually give you the curl approaches directly, and allow you to compare it to their API. Their oauth2 implementation is not hidden, and you can easily figure out what needs to be done.

Paypal
https://developer.paypal.com/docs/api/#authorizations
Pretty good outh2 implementation, well documented, easy to use API. Poor user object returned on successful auth in some cases.

Github
https://developer.github.com/v3/oauth/
Its good. Period.

Coinbase
https://www.coinbase.com/docs/api/authentication
Good implementation and documentation. The python examples were a little off but a python dev can figure it our. Their support responds to problems. Worthy.

Here are my recommendations to oauth providers:

• Make the entire three legged process easy to natively implement via curl commands.
• Build auth apis that are simple extensions of that flow 
• Make sure that the client api can access enough user information to create a user record on the target app without asking for more information. 
• Make it easy for developer to create client app secrets and manage multiple environments.

Apache 2 SSL Cert and Config

Intro

I am installing a SSL cert for Apache2 and I thought I would document it. I have done it so many times now and kinda feel like its about time to have a place to go to to get the details quickly. I am using a domain level cert because its cheap and will provide encryption without having to verify my "business" (there isnt one yet). The cert I am using for this is Comodo's positivessl cert from Namecheap.com that costs 9.99

Apache2 + mod_ssl

	#make sure the module is not there yet
	/usr/sbin/apachectl -t -D DUMP_MODULES
	#install ssl
	sudo yum install openssl-devel
	#get and build apache 2 based on your version
	cd src
	/usr/sbin/apachectl -v
	wget http://www.eng.lsu.edu/mirrors/apache//httpd/httpd-2.2.27.tar.gz
	tar xvfz httpd-2.2.27.tar.gz
	cd httpd-2.2.27
	./configure --enable-ssl --enable-so
	make
	sudo make install

view raw apache-ssl01.sh hosted with ❤ by GitHub

Configuration

	<VirtualHost gitpatron.com:443>
	ServerAdmin admin@gitpatron.com
	ServerName gitpatron.com
	SSLEngine on
	SSLCertificateKeyFile /etc/ssl/certs/gitpatron.key
	SSLCertificateFile /etc/ssl/certs/gitpatron_com.crt
	SSLCertificateChainFile /etc/ssl/certs/gitpatron_com.ca-bundle
	</VirtualHost>

view raw gistfile1.txt hosted with ❤ by GitHub

Sources

• The Geek Stuff - Installing SSL on Apache2
• Comodo CSR Generation
• Comodo Install Cert Apache2 mod_ssl
• Stack Overflow -Installing SSL on an EC2 Server

Actual Coinbase Client HTTP (Get & Post)

I like coinbase, seems like a pretty robust API, but sadly there are documentation bugs and they arent so hot at replying to support issues. After some thinking I figured out why the API calls were thinking they were not JSON.

	# there is a problem with the coinbase python example
	# at https://coinbase.com/docs/api/authentication where they
	# fail to add the content type header to the POST action
	# this is the actual implementation
	def get_http(self, url, body=None):
	opener = urllib2.build_opener()
	nonce = int(time.time() * 1e6)
	message = str(nonce) + url + ('' if body is None else body)
	signature = hmac.new(settings.COINBASE_API_SECRET, message, hashlib.sha256).hexdigest()
	opener.addheaders = [('ACCESS_KEY', settings.COINBASE_API_KEY),
	('ACCESS_SIGNATURE', signature),
	('ACCESS_NONCE', nonce)]
	try:
	response = opener.open(urllib2.Request(url,body,{'Content-Type': 'application/json'}))
	return response
	except urllib2.HTTPError as e:
	print e
	return e

view raw coinbase_http.py hosted with ❤ by GitHub

Using Mahout For Linear Regression

The goal here is to create the most simple of probability monitoring.

	package com.claytantor.mahout.test.app;
	import java.util.Collections;
	import java.util.HashMap;
	import java.util.List;
	import java.util.Map;
	import org.apache.mahout.classifier.sgd.L1;
	import org.apache.mahout.classifier.sgd.OnlineLogisticRegression;
	import org.apache.mahout.math.DenseVector;
	import org.apache.mahout.math.RandomAccessSparseVector;
	import org.apache.mahout.math.Vector;
	import org.slf4j.Logger;
	import org.slf4j.LoggerFactory;
	import com.google.common.collect.Lists;
	public class OnlineLROne {
	public static class Point {
	public int x;
	public int y;
	public Point(int x, int y) {
	this.x = x;
	this.y = y;
	}
	@Override
	public boolean equals(Object arg0) {
	Point p = (Point) arg0;
	return ((this.x == p.x) && (this.y == p.y));
	}
	@Override
	public String toString() {
	// TODO Auto-generated method stub
	return this.x + " , " + this.y;
	}
	}
	public static Vector getVector(Point point) {
	Vector v = new DenseVector(3);
	v.set(0, point.x);
	v.set(1, point.y);
	v.set(2, 1);
	return v;
	}
	/**
	* @param args
	*/
	public static void main(String[] args) {
	Logger logger = LoggerFactory.getLogger(ReccomenderOne.class);
	logger.info("Hello World");
	Map<Point, Integer> points = new HashMap<Point, Integer>();
	points.put(new Point(0, 0), 0);
	points.put(new Point(1, 1), 0);
	points.put(new Point(1, 0), 0);
	points.put(new Point(0, 1), 0);
	points.put(new Point(2, 2), 0);
	points.put(new Point(8, 8), 1);
	points.put(new Point(8, 9), 1);
	points.put(new Point(9, 8), 1);
	points.put(new Point(9, 9), 1);
	OnlineLogisticRegression learningAlgo = new OnlineLogisticRegression(2,
	3, new L1());
	// this is a really big value which will make the model very cautious
	// for lambda = 0.1, the first example below should be about .83 certain
	// for lambda = 0.01, the first example below should be about 0.98
	// certain
	learningAlgo.lambda(0.1);
	learningAlgo.learningRate(4);
	System.out.println("training model \n");
	final List<Point> keys = Lists.newArrayList(points.keySet());
	// 200 times through the training data is probably over-kill. It doesn't
	// matter
	// for tiny data. The key here is total number of points seen, not
	// number of passes.
	for (int i = 0; i < 200; i++) {
	// randomize training data on each iteration
	Collections.shuffle(keys);
	for (Point point : keys) {
	Vector v = getVector(point);
	learningAlgo.train(points.get(point), v);
	}
	}
	learningAlgo.close();
	// now classify real data
	Vector v = new RandomAccessSparseVector(3);
	v.set(0, 0.5);
	v.set(1, 0.5);
	v.set(2, 1);
	Vector r = learningAlgo.classifyFull(v);
	System.out.println(r);
	System.out.println("ans = ");
	System.out.printf("no of categories = %d\n",
	learningAlgo.numCategories());
	System.out.printf("no of features = %d\n", learningAlgo.numFeatures());
	System.out.printf("Probability of cluster 0 = %.3f\n", r.get(0));
	System.out.printf("Probability of cluster 1 = %.3f\n", r.get(1));
	v.set(0, 4.5);
	v.set(1, 6.5);
	v.set(2, 1);
	r = learningAlgo.classifyFull(v);
	System.out.println("ans = ");
	System.out.printf("no of categories = %d\n",
	learningAlgo.numCategories());
	System.out.printf("no of features = %d\n", learningAlgo.numFeatures());
	System.out.printf("Probability of cluster 0 = %.3f\n", r.get(0));
	System.out.printf("Probability of cluster 1 = %.3f\n", r.get(1));
	// show how the score varies along a line from 0,0 to 1,1
	System.out.printf("\nx\tscore\n");
	for (int i = 0; i < 100; i++) {
	final double x = 0.0 + i / 10.0;
	v.set(0, x);
	v.set(1, x);
	v.set(2, 1);
	r = learningAlgo.classifyFull(v);
	System.out.printf("%.2f\t%.3f\n", x, r.get(1));
	}
	}
	}

view raw Mahout LR Example hosted with ❤ by GitHub