Using OpenID/Keycloak with Superset

Question

I want to use keycloak to authenticate my users in our Superset environment.

Superset is using flask-openid, as implemented in flask-security:

• http://flask-appbuilder.readthedocs.io/en/latest/_modules/flask_appbuilder/security/manager.html
• https://pythonhosted.org/Flask-OpenID/

To enable a different user authentication than the regular one (database), you need to override the AUTH_TYPE parameter in your superset_config.py file. You will also need to provide a reference to your openid-connect realm and enable user registration. As I understand, it should look something like this:

from flask_appbuilder.security.manager import AUTH_OID
AUTH_TYPE = AUTH_OID
OPENID_PROVIDERS = [
    { 'name':'keycloak', 'url':'http://localhost:8080/auth/realms/superset' }
]
AUTH_USER_REGISTRATION = True
AUTH_USER_REGISTRATION_ROLE = 'Gamma'

With this configuration, the login page changes to a prompt where the user can select the desired OpenID provider (in our case keycloak). We also have two buttons, one to sign in (for existing users) and one to register as a new user.

I would expect that either of these buttons would take me to my keycloak login page. However, this does not happen. Instead, I am redirected right back to the login page.

In the case where I press the registration button, I get a message that says 'Not possible to register you at the moment, try again later'. When I press the sign in button, no message is displayed. The Superset logs show the request that loads the login page, but no requests to keycloak. I have tried the same using the Google OpenID provider, which works just fine.

Since I am seeing no requests to keycloak, this makes me think that I am either missing a configuration setting somewhere, or that I am using the wrong settings. Could you please help me figure out which settings I should be using?

Solution 1

Update 03-02-2020

@s.j.meyer has written an updated guide which works with Superset 0.28.1 and up. I haven't tried it myself, but thanks @nawazxy for confirming this solution works.

---

I managed to solve my own question. The main problem was caused by a wrong assumption I made regarding the flask-openid plugin that superset is using. This plugin actually supports OpenID 2.x, but not OpenID-Connect (which is the version implemented by Keycloak).

As a workaround, I decided to switch to the flask-oidc plugin. Switching to a new authentication provider actually requires some digging work. To integrate the plugin, I had to follow these steps:

Configue flask-oidc for keycloak

Unfortunately, flask-oidc does not support the configuration format generated by Keycloak. Instead, your configuration should look something like this:

{
    "web": {
        "realm_public_key": "<YOUR_REALM_PUBLIC_KEY>",
        "issuer": "http://<YOUR_DOMAIN>/auth/realms/<YOUR_REALM_ID>",
        "auth_uri": "http://<YOUR_DOMAIN>/auth/realms/<YOUR_REALM_ID>/protocol/openid-connect/auth",
        "client_id": "<YOUR_CLIENT_ID>",
        "client_secret": "<YOUR_SECRET_KEY>",
        "redirect_urls": [
            "http://<YOUR_DOMAIN>/*"
        ],
        "userinfo_uri": "http://<YOUR_DOMAIN>/auth/realms/<YOUR_REALM_ID>/protocol/openid-connect/userinfo",
        "token_uri": "http://<YOUR_DOMAIN>/auth/realms/<YOUR_REALM_ID>/protocol/openid-connect/token",
        "token_introspection_uri": "http://<YOUR_DOMAIN>/auth/realms/<YOUR_REALM_ID>/protocol/openid-connect/token/introspect"
    }
}

Flask-oidc expects the configuration to be in a file. I have stored mine in client_secret.json. You can configure the path to the configuration file in your superset_config.py.

Extend the Security Manager

Firstly, you will want to make sure that flask stops using flask-openid ad starts using flask-oidc instead. To do so, you will need to create your own security manager that configures flask-oidc as its authentication provider. I have implemented my security manager like this:

from flask_appbuilder.security.manager import AUTH_OID
from flask_appbuilder.security.sqla.manager import SecurityManager
from flask_oidc import OpenIDConnect
    
class OIDCSecurityManager(SecurityManager):

def __init__(self,appbuilder):
    super(OIDCSecurityManager, self).__init__(appbuilder)
    if self.auth_type == AUTH_OID:
        self.oid = OpenIDConnect(self.appbuilder.get_app)
    self.authoidview = AuthOIDCView

To enable OpenID in Superset, you would previously have had to set the authentication type to AUTH_OID. My security manager still executes all the behaviour of the super class, but overrides the oid attribute with the OpenIDConnect object. Further, it replaces the default OpenID authentication view with a custom one. I have implemented mine like this:

from flask_appbuilder.security.views import AuthOIDView
from flask_login import login_user
from urllib import quote

class AuthOIDCView(AuthOIDView):

@expose('/login/', methods=['GET', 'POST'])
def login(self, flag=True):
    
    sm = self.appbuilder.sm
    oidc = sm.oid

    @self.appbuilder.sm.oid.require_login
    def handle_login(): 
        user = sm.auth_user_oid(oidc.user_getfield('email'))
        
        if user is None:
            info = oidc.user_getinfo(['preferred_username', 'given_name', 'family_name', 'email'])
            user = sm.add_user(info.get('preferred_username'), info.get('given_name'), info.get('family_name'), info.get('email'), sm.find_role('Gamma')) 
        
        login_user(user, remember=False)
        return redirect(self.appbuilder.get_url_for_index)  
   
return handle_login()  

@expose('/logout/', methods=['GET', 'POST'])
def logout(self):
    
    oidc = self.appbuilder.sm.oid
    
    oidc.logout()
    super(AuthOIDCView, self).logout()        
    redirect_url = request.url_root.strip('/') + self.appbuilder.get_url_for_login
    
    return redirect(oidc.client_secrets.get('issuer') + '/protocol/openid-connect/logout?redirect_uri=' + quote(redirect_url))

My view overrides the behaviours at the /login and /logout endpoints. On login, the handle_login method is run. It requires the user to be authenticated by the OIDC provider. In our case, this means the user will first be redirected to Keycloak to log in.

On authentication, the user is redirected back to Superset. Next, we look up whether we recognize the user. If not, we create the user based on their OIDC user info. Finally, we log the user into Superset and redirect them to the landing page.

On logout, we will need to invalidate these cookies:

1. The superset session
2. The OIDC token
3. The cookies set by Keycloak

By default, Superset will only take care of the first. The extended logout method takes care of all three points.

Configure Superset

Finally, we need to add some parameters to our superset_config.py. This is how I've configured mine:

'''
AUTHENTICATION
'''
AUTH_TYPE = AUTH_OID
OIDC_CLIENT_SECRETS = 'client_secret.json'
OIDC_ID_TOKEN_COOKIE_SECURE = False
OIDC_REQUIRE_VERIFIED_EMAIL = False
CUSTOM_SECURITY_MANAGER = OIDCSecurityManager
AUTH_USER_REGISTRATION = True
AUTH_USER_REGISTRATION_ROLE = 'Gamma'

Solution 2

I had some trouble with the OIDC library, so I configured it a bit differently -

In Keycloak, I created a new client with standard flow and confidential access.
I also added a roles token claim in the mapper, so I could map "Client Roles" to Superset Roles.

For Superset, I mount the custom configuration files to my container [k8s in my case].

/app/pythonpath/custom_sso_security_manager.py

import logging
import os
import json
from superset.security import SupersetSecurityManager


logger = logging.getLogger('oauth_login')

class CustomSsoSecurityManager(SupersetSecurityManager):

    def oauth_user_info(self, provider, response=None):
        logging.debug("Oauth2 provider: {0}.".format(provider))

        logging.debug("Oauth2 oauth_remotes provider: {0}.".format(self.appbuilder.sm.oauth_remotes[provider]))

        if provider == 'keycloak':
            # Get the user info using the access token
            res = self.appbuilder.sm.oauth_remotes[provider].get(os.getenv('KEYCLOAK_BASE_URL') + '/userinfo')

            logger.info(f"userinfo response:")
            for attr, value in vars(res).items():
                print(attr, '=', value)

            if res.status_code != 200:
                logger.error('Failed to obtain user info: %s', res._content)
                return

            #dict_str = res._content.decode("UTF-8")
            me = json.loads(res._content)

            logger.debug(" user_data: %s", me)
            return {
                'username' : me['preferred_username'],
                'name' : me['name'],
                'email' : me['email'],
                'first_name': me['given_name'],
                'last_name': me['family_name'],
                'roles': me['roles'],
                'is_active': True,
            }

    def auth_user_oauth(self, userinfo):
        user = super(CustomSsoSecurityManager, self).auth_user_oauth(userinfo)
        roles = [self.find_role(x) for x in userinfo['roles']]
        roles = [x for x in roles if x is not None]
        user.roles = roles
        logger.debug(' Update <User: %s> role to %s', user.username, roles)
        self.update_user(user)  # update user roles
        return user

and in /app/pythonpath/superset_config.py I added some configs -

from flask_appbuilder.security.manager import AUTH_OAUTH, AUTH_REMOTE_USER

from custom_sso_security_manager import CustomSsoSecurityManager
CUSTOM_SECURITY_MANAGER = CustomSsoSecurityManager

oauthSecretPair = env('OAUTH_CLIENT_ID') + ':' + env('OAUTH_CLIENT_SECRET')

AUTH_TYPE = AUTH_OAUTH

OAUTH_PROVIDERS = [
    {   'name':'keycloak',
        'token_key':'access_token', # Name of the token in the response of access_token_url
        'icon':'fa-address-card',   # Icon for the provider
        'remote_app': {
            'api_base_url': env('KEYCLOAK_BASE_URL', 'http://CHANGEME'),
            'client_id':env('OAUTH_CLIENT_ID'),  # Client Id (Identify Superset application)
            'client_secret':env('OAUTH_CLIENT_SECRET'), # Secret for this Client Id (Identify Superset application)
            'client_kwargs':{
                'scope': 'profile'               # Scope for the Authorization
            },
            'request_token_url':None,
            'access_token_url': env('KEYCLOAK_BASE_URL', 'http://CHANGEME') + '/token',
            'authorize_url': env('KEYCLOAK_BASE_URL', 'http://CHANGEME') + '/auth',
        }
    }
]

# Will allow user self registration, allowing to create Flask users from Authorized User
AUTH_USER_REGISTRATION = True

# The default user self registration role
AUTH_USER_REGISTRATION_ROLE = "Gamma"

# This will make sure the redirect_uri is properly computed, even with SSL offloading
ENABLE_PROXY_FIX = True

There are a few env parameters that these configs expect -

KEYCLOAK_BASE_URL
OAUTH_CLIENT_ID
OAUTH_CLIENT_SECRET

Comments

[Kerneels Roos]

Hi, thanks a lot for this, but would you mind sharing a bit more info on how exactly one would go about doing the above, especially if working from an image such as amancevice/superset:0.24.0 plese?

[Kerneels Roos]

More specifically, where would one replace/override existing files, or is there perhaps a cleaner solution, like an updated package or system or image to use?

[thijsfranck]

I'm not sure how to do this when using Docker. We deployed our additions as part of a python module on the server that runs superset. Then we changed the superset_config.py as described above. My suggestion would be to have a look at the superset documentation on creating your custom build: superset.incubator.apache.org/…

[sj.meyer]

Hi @thijsfranck thank you for this solution! I'm trying to implement it, but running into a problem. I saved the security manager override code in a file called auth_manager.py and the view in auth_view.py, and stored both in the /security/ directory in the appbuilder package. When I run superset, it says it can't fine OIDCSecurityManager. Also tried putting it in /superset/, didn't work either. Is there a specific way to get the app to pick up the files, or a specific place to put it?

[thijsfranck]

For my project, I created a new package under the folder where my superset_config.py is stored. I put my code in the __init__.py file, but that's not a requirement. Depending on where you put your code, you will need to change your import in the superset_config.py file.

[ahmadalibaloch]

getting AttributeError: 'OpenIDConnect' object has no attribute 'loginhandler'

[ciolo]

Hi, if I have an user with Admin role how can I add/update the user's roles in db? In this way each user that log in has Gamma role, but I'd like to log in as an admin to manage database, dataset ecc.