Shibboleth¶

System Requirements¶

Support for Shibboleth in Dataverse is built on the popular “mod_shib” Apache module, “shibd” daemon, and the Embedded Discovery Service (EDS) Javascript library, all of which are distributed by the Shibboleth Consortium. EDS is bundled with Dataverse, but mod_shib and shibd must be installed and configured per below.

Only Red Hat Enterprise Linux (RHEL) 6 and derivatives such as CentOS have been tested (x86_64 versions) by the Dataverse team. Newer versions of RHEL and CentOS should work but you’ll need to adjust the yum repo config accordingly. See https://wiki.shibboleth.net/confluence/display/SHIB2/NativeSPLinuxInstall for details and note that (according to that page) as of this writing Ubuntu and Debian are not offically supported by the Shibboleth project.

Install Apache¶

We will be “fronting” Glassfish with Apache so that we can make use of the mod_shib Apache module. We will also make use of the mod_proxy_ajp module built in to Apache.

We include the mod_ssl package to enforce HTTPS per below.

yum install httpd mod_ssl

Install Shibboleth¶

Installing Shibboleth will give us both the shibd service and the mod_shib Apache module.

Configure Glassfish¶

Configure Apache¶

<VirtualHost *:80>

ServerName dataverse.example.edu

# From https://wiki.apache.org/httpd/RewriteHTTPToHTTPS

RewriteEngine On
# This will enable the Rewrite capabilities

RewriteCond %{HTTPS} !=on
# This checks to make sure the connection is not already HTTPS

RewriteRule ^/?(.*) https://%{SERVER_NAME}/$1 [R,L]
# This rule will redirect users from their original location, to the same location but using HTTPS.
# i.e.  http://www.example.com/foo/ to https://www.example.com/foo/
# The leading slash is made optional so that this will work either in httpd.conf
# or .htaccess context

</VirtualHost>

# don't pass paths used by rApache and TwoRavens to Glassfish
ProxyPassMatch ^/RApacheInfo$ !
ProxyPassMatch ^/custom !
ProxyPassMatch ^/dataexplore !
# don't pass paths used by Shibboleth to Glassfish
ProxyPassMatch ^/Shibboleth.sso !
ProxyPassMatch ^/shibboleth-ds !
# pass everything else to Glassfish
ProxyPass / ajp://localhost:8009/

<Location /shib.xhtml>
  AuthType shibboleth
  ShibRequestSetting requireSession 1
  require valid-user
</Location>

Configure Shibboleth¶

<!--
This is an example shibboleth2.xml generated originally by http://testshib.org
and tweaked for Dataverse.  See also:

- attribute-map.xml
- dataverse-idp-metadata.xml

https://wiki.shibboleth.net/confluence/display/SHIB2/NativeSPConfiguration
-->

<SPConfig xmlns="urn:mace:shibboleth:2.0:native:sp:config" xmlns:md="urn:oasis:names:tc:SAML:2.0:metadata"
    xmlns:saml="urn:oasis:names:tc:SAML:2.0:assertion"
    clockSkew="1800">

    <!-- FIXME: change the entityID to your hostname. -->
    <ApplicationDefaults entityID="https://dataverse.example.edu/sp"
        REMOTE_USER="eppn" attributePrefix="AJP_">

        <!-- You should use secure cookies if at all possible.  See cookieProps in this Wiki article. -->
        <!-- https://wiki.shibboleth.net/confluence/display/SHIB2/NativeSPSessions -->
        <Sessions lifetime="28800" timeout="3600" checkAddress="false" relayState="ss:mem" handlerSSL="false">

	    <SSO>
	      SAML2 SAML1
	    </SSO>

            <!-- SAML and local-only logout. -->
            <!-- https://wiki.shibboleth.net/confluence/display/SHIB2/NativeSPServiceLogout -->
            <Logout>SAML2 Local</Logout>

            <!--
                Handlers allow you to interact with the SP and gather more information.  Try them out!
                Attribute values received by the SP through SAML will be visible at:
                http://dataverse.example.edu/Shibboleth.sso/Session
            -->

            <!-- Extension service that generates "approximate" metadata based on SP configuration. -->
            <Handler type="MetadataGenerator" Location="/Metadata" signing="false"/>

            <!-- Status reporting service. -->
            <Handler type="Status" Location="/Status" acl="127.0.0.1"/>

            <!-- Session diagnostic service. -->
            <Handler type="Session" Location="/Session" showAttributeValues="true"/>

            <!-- JSON feed of discovery information. -->
            <Handler type="DiscoveryFeed" Location="/DiscoFeed"/>

        </Sessions>

        <!-- Error pages to display to yourself if something goes horribly wrong. -->
        <Errors supportContact="root@localhost" logoLocation="/shibboleth-sp/logo.jpg"
                styleSheet="/shibboleth-sp/main.css"/>

        <!-- Loads and trusts a metadata file that describes only the Testshib IdP and how to communicate with it. -->
        <!-- IdPs we want allow go in /etc/shibboleth/dataverse-idp-metadata.xml -->
        <MetadataProvider type="XML" file="dataverse-idp-metadata.xml" backingFilePath="local-idp-metadata.xml" legacyOrgNames="true" reloadInterval="7200"/>
        <!-- Uncomment to enable all the Research & Scholarship IdPs from InCommon -->
        <!--
        <MetadataProvider type="XML" url="http://md.incommon.org/InCommon/InCommon-metadata.xml" backingFilePath="InCommon-metadata.xml" maxRefreshDelay="3600">
            <DiscoveryFilter type="Whitelist" matcher="EntityAttributes">
                <saml:Attribute
                    Name="http://macedir.org/entity-category-support"
                    NameFormat="urn:oasis:names:tc:SAML:2.0:attrname-format:uri">
                    <saml:AttributeValue>http://id.incommon.org/category/research-and-scholarship</saml:AttributeValue>
                </saml:Attribute>
                <saml:Attribute
                    Name="http://macedir.org/entity-category-support"
                    NameFormat="urn:oasis:names:tc:SAML:2.0:attrname-format:uri">
                    <saml:AttributeValue>http://refeds.org/category/research-and-scholarship</saml:AttributeValue>
                </saml:Attribute>
            </DiscoveryFilter>
        </MetadataProvider>
        -->

        <!-- Attribute and trust options you shouldn't need to change. -->
        <AttributeExtractor type="XML" validate="true" path="attribute-map.xml"/>
        <AttributeResolver type="Query" subjectMatch="true"/>
        <AttributeFilter type="XML" validate="true" path="attribute-policy.xml"/>

        <!-- Your SP generated these credentials.  They're used to talk to IdP's. -->
        <CredentialResolver type="File" key="sp-key.pem" certificate="sp-cert.pem"/>

    </ApplicationDefaults>

    <!-- Security policies you shouldn't change unless you know what you're doing. -->
    <SecurityPolicyProvider type="XML" validate="true" path="security-policy.xml"/>

    <!-- Low-level configuration about protocols and bindings available for use. -->
    <ProtocolProvider type="XML" validate="true" reloadChanges="false" path="protocols.xml"/>

</SPConfig>

Disable SELinux¶

You must set SELINUX=permisive in /etc/selinux/config and run setenforce permissive or otherwise disable SELinux for Shibboleth to work. “At the present time, we do not support the SP in conjunction with SELinux, and at minimum we know that communication between the mod_shib and shibd components will fail if it’s enabled. Other problems may also occur.” – https://wiki.shibboleth.net/confluence/display/SHIB2/NativeSPSELinux

Restart Apache and Shibboleth¶

After configuration is complete:

service shibd restart

service httpd restart

Configure Apache and shibd to Start at Boot¶

chkconfig httpd on

chkconfig shibd on

Verify DiscoFeed and Metadata URLs¶

As a sanity check, visit the following URLs (substituting your hostname) to make sure you see JSON and XML:

• https://dataverse.example.edu/Shibboleth.sso/DiscoFeed
• https://dataverse.example.edu/Shibboleth.sso/Metadata

The JSON in DiscoFeed comes from the list of IdPs you configured in the MetadataProvider section of shibboleth2.xml and will form a dropdown list on the Login Page.

Enable Shibboleth¶

curl -X PUT -d true http://localhost:8080/api/admin/settings/:ShibEnabled

After enabling Shibboleth, assuming the DiscoFeed is working per above, you should see a list of institutions to log into. You will not be able to log in via these institutions, however, until you have exchanged metadata with them. You can change the boolean above to false while you wait for the metadata exchange to be complete since it only affects if the Shibboleth login screen is shown.

Exchange Metadata with Your Identity Provider¶

http://testshib.org (TestShib) is a fantastic resource for testing Shibboleth configurations. Depending on your relationship with your identity people you may want to avoid bothering them until you have tested your Dataverse configuration with the TestShib Identity Provider (IdP). This process is explained below.

If you’ve temporarily configured your MetadataProvider to use the TestShib Identity Provider (IdP) as outlined above, you can download your metadata like this (substituting your hostname in both places):

curl https://dataverse.example.edu/Shibboleth.sso/Metadata > dataverse.example.edu

Then upload your metadata to http://testshib.org/register.html

Then try to log in to Dataverse using the TestShib IdP. After logging in, you can visit the https://dataverse.example.edu/Shibboleth.sso/Session (substituting your hostname) to troubleshoot which attributes are being received. You should see something like the following:

Miscellaneous
Session Expiration (barring inactivity): 479 minute(s)
Client Address: 65.112.10.82
SSO Protocol: urn:oasis:names:tc:SAML:2.0:protocol
Identity Provider: https://idp.testshib.org/idp/shibboleth
Authentication Time: 2016-03-08T13:45:10.922Z
Authentication Context Class: urn:oasis:names:tc:SAML:2.0:ac:classes:PasswordProtectedTransport
Authentication Context Decl: (none)

Attributes
affiliation: Member@testshib.org;Staff@testshib.org
cn: Me Myself And I
entitlement: urn:mace:dir:entitlement:common-lib-terms
eppn: myself@testshib.org
givenName: Me Myself
persistent-id: https://idp.testshib.org/idp/shibboleth!https://dataverse.example.edu/sp!RuyCiLvUcgmKqyh/rOQPh+wyR7s=
sn: And I
telephoneNumber: 555-5555
uid: myself
unscoped-affiliation: Member;Staff

(As of this writing the TestShib IdP does not send the “mail” attribute, a required attribute, but for testing purposes, Dataverse compensates for this for the TestShib IdP and permits login anyway.)

When you are done testing, you can delete the TestShib users you created like this (after you have deleted any data and permisions associated with the users):

curl -X DELETE http://localhost:8080/api/admin/authenticatedUsers/myself

(Of course, you are also welcome to do a fresh reinstall per the Installation section.)

If your Dataverse installation is working with TestShib it should work with your institution’s Identity Provider (IdP). Next, you should:

• Send your identity people your metadata file above (or a link to download it themselves). From their perspective you are a Service Provider (SP).
• Ask your identity people to send you the metadata for the Identity Provider (IdP) they operate. See the section above on shibboleth2.xml and MetadataProvider for what to do with the IdP metadata. Restart shibd and httpd as necessary.
• Test login to Dataverse via your institution’s Identity Provider (IdP).

Institution-Wide Shibboleth Groups¶

Dataverse allows you to optionally define “institution-wide Shibboleth groups” based on the the entityID of the Identity Provider (IdP) used to authenticate. For example, an “institution-wide Shibboleth group” with https://idp.testshib.org/idp/shibboleth as the IdP would include everyone who logs in via the TestShib IdP mentioned above.

To create an institution-wide Shibboleth groups, create a JSON file as below and issue this curl command: curl http://localhost:8080/api/admin/groups/shib -X POST -H 'Content-type:application/json' --upload-file shibGroupTestShib.json

{
  "name": "All testshib.org Shibboleth Users",
  "attribute": "Shib-Identity-Provider",
  "pattern": "https://idp.testshib.org/idp/shibboleth"
}

Institution-wide Shibboleth groups are based on the “Shib-Identity-Provider” SAML attribute asserted at runtime after successful authentication with the Identity Provider (IdP) and held within the browser session rather than being persisted in the database for any length of time. It is for this reason that roles based on these groups, such as the ability to create a dataset, are not honored by non-browser interactions, such as through the SWORD API.

To list institution-wide Shibboleth groups: curl http://localhost:8080/api/admin/groups/shib

To delete an institution-wide Shibboleth group (assuming id 1): curl -X DELETE http://localhost:8080/api/admin/groups/shib/1

Support for arbitrary attributes beyond “Shib-Identity-Provider” such as “eduPersonScopedAffiliation”, etc. is being tracked at https://github.com/IQSS/dataverse/issues/1515

Converting Local Users to Shibboleth¶

If you are running in “remote and local” mode and have existing local users that you’d like to convert to Shibboleth users, give them the following steps to follow:

• Log in with your local account to make sure you know your password, which will be needed for the account conversion process.
• Log out of your local account.
• Log in with your Shibboleth account.
• If the email address associated with your local account matches the email address asserted by the Identity Provider (IdP), you will be prompted for the password of your local account and asked to confirm the conversion of your account. You’re done! Browse around to ensure you see all the data you expect to see. Permissions have been preserved.
• If the email address asserted by the Identity Provider (IdP) does not match the email address of any local user, you will be prompted to create a new account. If you were expecting account conversion, you should decline creating a new Shibboleth account, log back in to your local account, and let Support know the email on file for your local account. Support may ask you to change your email address for your local account to the one that is being asserted by the Identity Provider. Someone with access to the Glassfish logs will see this email address there.

Converting Shibboleth Users to Local¶

Whereas users convert their own accounts from local to Shibboleth as described above, conversion in the opposite direction is performed by a sysadmin. A common scenario may be as follows:

• A user emails Support saying, “I left the university (or wherever) and can’t log in to Dataverse anymore. What should I do?”
• Support replies asking the user for a new email address (Gmail, new institution email, etc.) to associate with their Dataverse account.
• The user replies with a new email address to associate with their Dataverse account.
• Support runs the curl command below, supplying the database id of the user to convert and the new email address and notes the username returned.
• Support emails the user and indicates that that they should use the password reset feature to set a new password and to make sure to take note of their username under Account Information (or the password reset confirmation email) since the user never had a username before.
• The user resets password and is able to log in with their local account. All permissions have been preserved with the exception of any permissions assigned to an institution-wide Shibboleth group to which the user formerly belonged.

In the example below, the user has indicated that the new email address they’d like to have associated with their account is “former.shib.user@mailinator.com” and their user id from the authenticateduser database table is “2”. The API token must belong to a superuser (probably the sysadmin executing the command).

curl -H "X-Dataverse-key: $API_TOKEN" -X PUT -d "former.shib.user@mailinator.com" http://localhost:8080/api/admin/authenticatedUsers/id/2/convertShibToBuiltIn

Rather than looking up the user’s id in the authenticateduser database table, you can issue this command to get a listing of all users:

curl -H "X-Dataverse-key: $API_TOKEN" http://localhost:8080/api/admin/authenticatedUsers

Per above, you now need to tell the user to use the password reset feature to set a password for their local account.

Debugging¶

The Administration section explains how to increase Glassfish logging levels. The relevant classes and packages are:

• edu.harvard.iq.dataverse.Shib
• edu.harvard.iq.dataverse.authorization.providers.shib
• edu.harvard.iq.dataverse.authorization.groups.impl.shib

Backups¶

The installation and configuration of Shibboleth will result in the following cert and key files being created and it’s important to back them up:

• /etc/shibboleth/sp-cert.pem
• /etc/shibboleth/sp-key.pem

If you have more than one Glassfish server, you should use the same sp-cert.pem and sp-key.pem files on all of them. If these files are compromised and you need to regenerate them, you can cd /etc/shibboleth and run keygen.sh like this (substituting you own hostname):

./keygen.sh -f -u shibd -g shibd -h dataverse.example.edu -e https://dataverse.example.edu/sp