Shibboleth¶

System Requirements¶

Support for Shibboleth in Dataverse is built on the popular “mod_shib” Apache module, “shibd” daemon, and the Embedded Discovery Service (EDS) Javascript library, all of which are distributed by the Shibboleth Consortium. EDS is bundled with Dataverse, but mod_shib and shibd must be installed and configured per below.

Only Red Hat Enterprise Linux (RHEL) and derivatives such as CentOS have been tested (x86_64 versions) by the Dataverse team. See https://wiki.shibboleth.net/confluence/display/SHIB2/NativeSPLinuxInstall for details and note that (according to that page) as of this writing Ubuntu and Debian are not offically supported by the Shibboleth project.

Install Apache¶

We will be “fronting” Glassfish with Apache so that we can make use of the mod_shib Apache module. We will also make use of the mod_proxy_ajp module built in to Apache.

We include the mod_ssl package to enforce HTTPS per below.

yum install httpd mod_ssl

Install Shibboleth¶

Installing Shibboleth will give us both the shibd service and the mod_shib Apache module.

Apply GRIZZLY-1787 Patch¶

In order for the Dataverse “download as zip” feature to work well with large files without causing OutOfMemoryError problems on Glassfish 4.1 when fronted with Apache, you should stop Glassfish, with ./asadmin stop-domain domain1, make a backup of glassfish4/glassfish/modules/glassfish-grizzly-extra-all.jar, replace it with a patched version of glassfish-grizzly-extra-all.jar downloaded from here (the md5 is in the README), and start Glassfish again with ./asadmin start-domain domain1.

For more background on the patch, please see https://java.net/jira/browse/GRIZZLY-1787 and https://github.com/IQSS/dataverse/issues/2180 and https://github.com/payara/Payara/issues/350

This problem has been reported to Glassfish at https://java.net/projects/glassfish/lists/users/archive/2015-07/message/1 and while Glassfish 4.1.1 includes a new enough version of Grizzly to fix the bug, other complicating factors prevent its adoption (look for “Glassfish 4.1.1” in the Prerequisites section for details on why it is not recommended).

Glassfish HTTP and HTTPS ports¶

Apache will be listening on ports 80 and 443 so we need to make sure Glassfish isn’t using them. If you’ve been changing the default ports used by Glassfish per the Configuration section, revert the Glassfish HTTP service to listen on 8080, the default port:

./asadmin set server-config.network-config.network-listeners.network-listener.http-listener-1.port=8080

Likewise, if necessary, revert the Glassfish HTTPS service to listen on port 8181:

./asadmin set server-config.network-config.network-listeners.network-listener.http-listener-2.port=8181

AJP¶

A jk-connector network listener should have already been set up when you ran the installer mentioned in the Installation section, but for reference, here is the command that is used:

./asadmin create-network-listener --protocol http-listener-1 --listenerport 8009 --jkenabled true jk-connector

You can verify this with ./asadmin list-network-listeners.

This enables the AJP protocol used in Apache configuration files below.

SSLEngine Warning Workaround¶

When fronting Glassfish with Apache and using the jk-connector (AJP, mod_proxy_ajp), in your Glassfish server.log you can expect to see “WARNING ... org.glassfish.grizzly.http.server.util.RequestUtils ... jk-connector ... Unable to populate SSL attributes java.lang.IllegalStateException: SSLEngine is null”.

To hide these warnings, run ./asadmin set-log-levels org.glassfish.grizzly.http.server.util.RequestUtils=SEVERE so that the WARNING level is hidden as recommended at https://java.net/jira/browse/GLASSFISH-20694 and https://github.com/IQSS/dataverse/issues/643#issuecomment-49654847

Enforce HTTPS¶

To prevent attacks such as FireSheep, HTTPS should be enforced. https://wiki.apache.org/httpd/RewriteHTTPToHTTPS provides a good method. You could copy and paste that those “rewrite rule” lines into Apache’s main config file at /etc/httpd/conf/httpd.conf but using Apache’s “virtual hosts” feature is recommended so that you can leave the main configuration file alone and drop a host-specific file into place.

Below is an example of how “rewrite rule” lines look within a VirtualHost block. Download a sample file , edit it to substitute your own hostname under ServerName, and place it at /etc/httpd/conf.d/dataverse.example.edu.conf or a filename that matches your hostname. The file must be in /etc/httpd/conf.d and must end in ”.conf” to be included in Apache’s configuration.

<VirtualHost *:80>

ServerName dataverse.example.edu

# From https://wiki.apache.org/httpd/RewriteHTTPToHTTPS

RewriteEngine On
# This will enable the Rewrite capabilities

RewriteCond %{HTTPS} !=on
# This checks to make sure the connection is not already HTTPS

RewriteRule ^/?(.*) https://%{SERVER_NAME}/$1 [R,L]
# This rule will redirect users from their original location, to the same location but using HTTPS.
# i.e.  http://www.example.com/foo/ to https://www.example.com/foo/
# The leading slash is made optional so that this will work either in httpd.conf
# or .htaccess context

</VirtualHost>

Edit Apache ssl.conf File¶

/etc/httpd/conf.d/ssl.conf should be edited to contain the FQDN of your hostname like this: ServerName dataverse.example.edu:443 (substituting your hostname).

Near the bottom of /etc/httpd/conf.d/ssl.conf but before the closing </VirtualHost> directive, add the following:

# don't pass paths used by rApache and TwoRavens to Glassfish
ProxyPassMatch ^/RApacheInfo$ !
ProxyPassMatch ^/custom !
ProxyPassMatch ^/dataexplore !
# don't pass paths used by Shibboleth to Glassfish
ProxyPassMatch ^/Shibboleth.sso !
ProxyPassMatch ^/shibboleth-ds !
# pass everything else to Glassfish
ProxyPass / ajp://localhost:8009/

<Location /shib.xhtml>
  AuthType shibboleth
  ShibRequestSetting requireSession 1
  require valid-user
</Location>

You can download a sample ssl.conf file to compare it against the file you edited.

Note that /etc/httpd/conf.d/shib.conf and /etc/httpd/conf.d/shibboleth-ds.conf are expected to be present from installing Shibboleth via yum.

You may wish to also add a timeout directive to the ProxyPass line within ssl.conf. This is especially useful for larger file uploads as apache may prematurely kill the connection before the upload is processed.

e.g. ProxyPass / ajp://localhost:8009/ timeout=600 defines a timeout of 600 seconds.

Try to strike a balance with the timeout setting. Again a timeout too low will impact file uploads. A timeout too high may cause additional stress on the server as it will have to service idle clients for a longer period of time.

shibboleth2.xml¶

/etc/shibboleth/shibboleth2.xml should look something like the sample shibboleth2.xml file below, but you must substitute your hostname in the entityID value. If your starting point is a shibboleth2.xml file provided by someone else, you must ensure that attributePrefix="AJP_" is added under ApplicationDefaults per the Shibboleth wiki . Without the AJP_ configuration in place, the required Shibboleth Attributes will be null and users will be unable to log in.

<!--
This is an example shibboleth2.xml generated originally by http://testshib.org
and tweaked for Dataverse.  See also:

- attribute-map.xml
- dataverse-idp-metadata.xml

https://wiki.shibboleth.net/confluence/display/SHIB2/NativeSPConfiguration
-->

<SPConfig xmlns="urn:mace:shibboleth:3.0:native:sp:config" xmlns:md="urn:oasis:names:tc:SAML:2.0:metadata"
    xmlns:saml="urn:oasis:names:tc:SAML:2.0:assertion"
    clockSkew="1800">

    <!-- FIXME: change the entityID to your hostname. -->
    <ApplicationDefaults entityID="https://dataverse.example.edu/sp"
        REMOTE_USER="eppn" attributePrefix="AJP_">

        <!-- You should use secure cookies if at all possible.  See cookieProps in this Wiki article. -->
        <!-- https://wiki.shibboleth.net/confluence/display/SHIB2/NativeSPSessions -->
        <Sessions lifetime="28800" timeout="3600" checkAddress="false" relayState="ss:mem" handlerSSL="false">

	    <SSO>
	      SAML2 SAML1
	    </SSO>

            <!-- SAML and local-only logout. -->
            <!-- https://wiki.shibboleth.net/confluence/display/SHIB2/NativeSPServiceLogout -->
            <Logout>SAML2 Local</Logout>

            <!--
                Handlers allow you to interact with the SP and gather more information.  Try them out!
                Attribute values received by the SP through SAML will be visible at:
                http://dataverse.example.edu/Shibboleth.sso/Session
            -->

            <!-- Extension service that generates "approximate" metadata based on SP configuration. -->
            <Handler type="MetadataGenerator" Location="/Metadata" signing="false"/>

            <!-- Status reporting service. -->
            <Handler type="Status" Location="/Status" acl="127.0.0.1"/>

            <!-- Session diagnostic service. -->
	    <!-- showAttributeValues must be set to true to see attributes at /Shibboleth.sso/Session . -->
            <Handler type="Session" Location="/Session" showAttributeValues="true"/>

            <!-- JSON feed of discovery information. -->
            <Handler type="DiscoveryFeed" Location="/DiscoFeed"/>

        </Sessions>

        <!-- Error pages to display to yourself if something goes horribly wrong. -->
        <Errors supportContact="root@localhost" logoLocation="/shibboleth-sp/logo.jpg"
                styleSheet="/shibboleth-sp/main.css"/>

        <!-- Loads and trusts a metadata file that describes only the Testshib IdP and how to communicate with it. -->
        <!-- IdPs we want allow go in /etc/shibboleth/dataverse-idp-metadata.xml -->
        <MetadataProvider type="XML" path="dataverse-idp-metadata.xml" backingFilePath="local-idp-metadata.xml" legacyOrgNames="true" reloadInterval="7200"/>
        <!-- Uncomment to enable all the Research & Scholarship IdPs from InCommon -->
        <!--
        <MetadataProvider type="XML" url="http://md.incommon.org/InCommon/InCommon-metadata.xml" backingFilePath="InCommon-metadata.xml" maxRefreshDelay="3600">
            <DiscoveryFilter type="Whitelist" matcher="EntityAttributes">
                <saml:Attribute
                    Name="http://macedir.org/entity-category-support"
                    NameFormat="urn:oasis:names:tc:SAML:2.0:attrname-format:uri">
                    <saml:AttributeValue>http://id.incommon.org/category/research-and-scholarship</saml:AttributeValue>
                </saml:Attribute>
                <saml:Attribute
                    Name="http://macedir.org/entity-category-support"
                    NameFormat="urn:oasis:names:tc:SAML:2.0:attrname-format:uri">
                    <saml:AttributeValue>http://refeds.org/category/research-and-scholarship</saml:AttributeValue>
                </saml:Attribute>
            </DiscoveryFilter>
        </MetadataProvider>
        -->

        <!-- Attribute and trust options you shouldn't need to change. -->
        <AttributeExtractor type="XML" validate="true" path="attribute-map.xml"/>
        <AttributeResolver type="Query" subjectMatch="true"/>
        <AttributeFilter type="XML" validate="true" path="attribute-policy.xml"/>

        <!-- Your SP generated these credentials.  They're used to talk to IdP's. -->
        <CredentialResolver type="File" key="sp-key.pem" certificate="sp-cert.pem"/>

    </ApplicationDefaults>

    <!-- Security policies you shouldn't change unless you know what you're doing. -->
    <SecurityPolicyProvider type="XML" validate="true" path="security-policy.xml"/>

    <!-- Low-level configuration about protocols and bindings available for use. -->
    <ProtocolProvider type="XML" validate="true" reloadChanges="false" path="protocols.xml"/>

</SPConfig>

Shibboleth Attributes¶

The following attributes are required for a successful Shibboleth login:

• Shib-Identity-Provider
• eppn
• givenName
• sn
• email

See also https://www.incommon.org/federation/attributesummary.html and https://wiki.shibboleth.net/confluence/display/SHIB2/NativeSPAttributeAccess

attribute-map.xml¶

By default, some attributes /etc/shibboleth/attribute-map.xml are commented out and “subject-id” is used instead of “eppn”. We recommend downloading and using attribute-map.xml instead which has these changes and should be compatible with Dataverse.

Shibboleth and ADFS¶

With appropriate configuration, Dataverse and Shibboleth can make use of “single sign on” using Active Directory. This requires configuring shibd and httpd to load appropriate libraries, and insuring that the attribute mapping matches those provided. Example configuration files for shibboleth2.xml and attribute-map.xml may be helpful. Note that your ADFS server hostname goes in the file referenced under “MetadataProvider” in your shibboleth2.xml file.

Disable SELinux¶

The first and easiest option is to set SELINUX=permisive in /etc/selinux/config and run setenforce permissive or otherwise disable SELinux to get Shibboleth to work. This is apparently what the Shibboleth project expects because their wiki page at https://wiki.shibboleth.net/confluence/display/SHIB2/NativeSPSELinux says, “At the present time, we do not support the SP in conjunction with SELinux, and at minimum we know that communication between the mod_shib and shibd components will fail if it’s enabled. Other problems may also occur.”

Reconfigure SELinux to Accommodate Shibboleth¶

The second (more involved) option is to use the checkmodule, semodule_package, and semodule tools to apply a local policy to make Shibboleth work with SELinux. Let’s get started.

module shibboleth 1.0;

require {
    class file {open read};
    class sock_file write;
    class unix_stream_socket connectto;
    type httpd_t;
    type initrc_t;
    type var_run_t;
    type var_t;
}

allow httpd_t initrc_t:unix_stream_socket connectto;
allow httpd_t var_run_t:sock_file write;
allow httpd_t var_t:file {open read};

Converting Local Users to Shibboleth¶

If you are running in “remote and local” mode and have existing local users that you’d like to convert to Shibboleth users, give them the following steps to follow, which are also explained in the Account Creation + Management section of the User Guide:

• Log in with your local account to make sure you know your password, which will be needed for the account conversion process.
• Log out of your local account.
• Log in with your Shibboleth account.
• If the email address associated with your local account matches the email address asserted by the Identity Provider (IdP), you will be prompted for the password of your local account and asked to confirm the conversion of your account. You’re done! Browse around to ensure you see all the data you expect to see. Permissions have been preserved.
• If the email address asserted by the Identity Provider (IdP) does not match the email address of any local user, you will be prompted to create a new account. If you were expecting account conversion, you should decline creating a new Shibboleth account, log back in to your local account, and let Support know the email on file for your local account. Support may ask you to change your email address for your local account to the one that is being asserted by the Identity Provider. Someone with access to the Glassfish logs will see this email address there.

Converting Shibboleth Users to Local¶

Whereas users convert their own accounts from local to Shibboleth as described above, conversion in the opposite direction is performed by a sysadmin. A common scenario may be as follows:

• A user emails Support saying, “I left the university (or wherever) and can’t log in to Dataverse anymore. What should I do?”
• Support replies asking the user for a new email address (Gmail, new institution email, etc.) to associate with their Dataverse account.
• The user replies with a new email address to associate with their Dataverse account.
• Support runs the curl command below, supplying the database id of the user to convert and the new email address and notes the username returned.
• Support emails the user and indicates that that they should use the password reset feature to set a new password and to make sure to take note of their username under Account Information (or the password reset confirmation email) since the user never had a username before.
• The user resets password and is able to log in with their local account. All permissions have been preserved with the exception of any permissions assigned to an institution-wide Shibboleth group to which the user formerly belonged.

In the example below, the user has indicated that the new email address they’d like to have associated with their account is “former.shib.user@mailinator.com” and their user id from the authenticateduser database table is “2”. The API token must belong to a superuser (probably the sysadmin executing the command). Note that the old version of this call, convertShibToBuiltIn, is deprecated and will be deleted in a future release.

curl -H "X-Dataverse-key: $API_TOKEN" -X PUT -d "former.shib.user@mailinator.com" http://localhost:8080/api/admin/authenticatedUsers/id/2/convertRemoteToBuiltIn

Rather than looking up the user’s id in the authenticateduser database table, you can issue this command to get a listing of all users:

curl -H "X-Dataverse-key: $API_TOKEN" http://localhost:8080/api/admin/authenticatedUsers

Per above, you now need to tell the user to use the password reset feature to set a password for their local account.