Annex 3: Executive summary of the project entitled “Accessibility of online job application and recruitment systems”

Introduction

The goal of the project entitled “Accessibility of online job application and recruitment systems” was to assess the online job application and recruitment systems and related content of the following United Nations entities:

	International Labour Organization (ILO)
	United Nations High Commissioner for Refugees (UNHCR)
	World Health Organization (WHO)
	United Nations Secretariat
	World Intellectual Property Organization (WIPO)

The British non-governmental organization Leonard Cheshire also participated in the project.

The aims of the project were to determine:

•	whether the entire online job application process of each entity conformed to the success criteria for level AA of WCAG 2.1;
•	whether critical steps in the online job application and recruitment system process were accessible, including by:

	º	ensuring that all online recruitment forms and/or digital content was available in accessible formats;
	º	ensuring that upload processes were fully accessible to applicants with disabilities who wished to present documents as part of their candidacy, such as résumés or certifications;
	º	preventing accessibility and usability barriers that could lead to online application drop-off (such as consistency, flexibility and the time allowed for completion);

•	whether the entire online job application process of each entity provided an overall accessible and positive experience for end users.

To this end, an accessibility audit was performed, consisting of an in-depth evaluation of a set of pages on each organization’s website or web/desktop application with a view to documenting the types of accessibility issues experienced and any violations of level AA compliance with WCAG 2.1 and providing detailed recommendations on how to remedy the violations. For the accessibility audit, the evaluator used a combination of accessibility evaluation tools, screen readers and browsers. These audits not only highlighted the key violations, but also provided accessibility remediation suggestions and feedback such as user experience issues.

In addition, remote user testing sessions were performed with the involvement of persons with different types of impairments in order to determine the extent to which a given interface facilitated users’ ability to complete specific tasks. Collaboration with Spanish organization Fundación ONCE and with UNHCR was instrumental for recruiting testers. Sessions took place remotely using web conferencing software platforms. During the sessions, users were asked to complete a series of tasks. The sessions were recorded and analyzed to identify potential areas for improvement on each website. The purpose of these testing sessions was to assess the general usability of the site’s web interface design, information flow and architecture, services and functionalities by collecting relevant data on user experiences of carrying out different tasks on the job portal using a computer, smartphone or tablet, across various operating systems and browsers and with a variety of assistive technologies (where required). Any barriers, difficulties or commendable aspects encountered by users while performing these tasks were identified and described, in addition to the feelings and perceptions that they triggered in users.

Individual reports with findings and recommendations were issued for each organization following the completion of the accessibility audit and user testing.

This document is a general report summarizing the most common findings included in the individual reports.

Summary of accessibility audit results

The following is a list of the key findings. They primarily apply to desktop users, but some also apply to mobile users. The findings are organized by WCAG principle, issue type and their impact on persons with disabilities.

Perceivable

Information and user interface components must be presented to users in ways that they can perceive. This means that users must be able to perceive the information presented. In other words, the content cannot be invisible to all the user’s senses.

Key issues found:

•	Alternative text: Also known as “alt text”, “alt attributes” or “alt descriptions”, alternative text is used within the HTML code of a web page to provide a short-written description of the appearance and function of an image. Alternative text was found to be missing from some image elements (banners, job opportunity PDF images, social media active elements in the footers of websites, etc.) on the pages of some portals.
•	Headings: Headings provide structure to web page content by breaking information down into ordered sections. A heading describes the content that will follow it, similar to a news headline. When sighted users arrive at a new page, they can use the headings to quickly find the content that they are looking for on the page. However, headings are fundamental for blind, visually impaired or dyslexic persons who use screen reading software and other assistive technologies to skip from heading to heading and navigate web pages in an agile manner. When headings are clear and descriptive, users can more easily find the information that they seek and understand the relationships between different parts of the content. Without headings, users are unable to navigate to specific sections and will likely leave the website. On several pages of the portals of the organizations assessed, headings were found to be either missing or wrongly designated as level one (“H1” in the HTML code of the website). Consequently, the headers did not describe the content or purpose of the page, the page appeared to contain more than one level one heading, and the heading level broke the hierarchical order, all of which can confuse assistive technology users.
•	Use of colour to convey information: On some pages, the current step on a progress bar was indicated only visually, through colour, rather than programmatically.
•	The contrast between text colour and background colours was insufficient.
•	Some search fields lacked an associated form label
•	Navigation lists were not defined as ordered lists. Where lists are not defined, non-visual users have no way to know how many items are in the list and may find it harder to navigate through the items.
•	Content reflow: On some pages (especially job search pages) content was presented at a width of 320 pixels, forcing users to scroll both horizontally and vertically to read content and interact with controls. Low vision users who need to enlarge text were not able to view and read the content in a single column, forcing them to scroll in the direction of reading to reveal lines that were cut off by the viewport. This requires more effort from the users’ side to read the same amount of content.
•	Large blocks of justified text were used, which could have a negative impact on document readability.
•	Input field purpose: Input fields for collecting certain types of user information (e.g. first and last names and date of birth) lacked an autocomplete definition visible to assistive technologies. Autocomplete options make it is easier for persons with disabilities to fill out input fields where a specific data type is expected.
•	Adequate semantics/tags were missing from PDF documents, including:

	º	missing or inadequate titles
	º	inadequate descriptive text on images
	º	no additional information on complex images
	º	no relationship between headings and data cells in tables.

•	Videos lacked accessible names recognizable by assistive technologies.

Operable

User interface components and navigation must be operable. This means that users must be able to operate the interface using only a keyboard. In other words, the interface must not require interactions that users cannot perform.

Key issues found:

•	In-page navigation: Some pages lacked a skip link to allow users to skip over a block of links or text to reach the main content area, thereby making page navigation more difficult for keyboard-only users.
•	Page title: Some pages (such as job search) had an HTML title element that was either not meaningful enough or not descriptive enough. Page titles are important as they provide assistive technology users with orientation information when the page initially loads.
•	Link texts:

	º	Generic link text (e.g. “Click here”) was used. Such text should be avoided in links, as it does not transmit clear information to assistive technology users.
	º	Some empty links were given keyboard focus, which can be confusing for assistive technology users.
	º	Some links were not placed far enough apart, which could cause users to click the wrong link.

•	In some pages, content in tables was structured poorly.
•	Visible focus:

	º	Some active elements had no visible focus indicator. Mouse users were given visible cues that a page element was active, but these cues were not available to keyboard-only users.
	º	Elements were located on a header before the main menu without a visible focus indicator, making them invisible to keyboard-only users.

•	Keyboard access:

	º	In some job search pages, there was no indication to let keyboard-only users know that they could collapse or expand a group of search filters.
	º	The “Play” button on embedded video players was not accessible to keyboard-only users.
	º	When filling out a user information form, the only way to select a country was by clicking on the corresponding country flag with a mouse.

Understandable

Information and the operation of the user interface must be understandable. This means that users must be able to understand both the information on the web page and how to operate the user interface. In other words, neither the content nor the method of operation should be beyond users’ understanding.

Key issues found:

•	Lack of a defined language
•	Lack of visible focus indicator, which makes navigation extremely difficult.
•	The language selection button could not be read by screen reading software.
•	Error handling in job application or new user forms:

	º	In some job application forms, mandatory fields were marked visibly with an asterisk but were not marked programmatically.
	º	Error messages at the top of the form were defined but lacked a heading to identify the content as errors.
	º	Error messages were not as coherent and brief as possible. Long messages are difficult for screen reader users to listen to.
	º	Error messages on job application pages were not announced by the screen reader on either the desktop or smartphone version of the web page.
	º	In case of password-related errors, password rules that were already defined on the page were repeated at the top of the form.

Robust

Content must be robust enough so that a wide variety of user agents can interpret it reliably, including assistive technologies.

Key issues found:

•

In forms:

	º	No mechanism was available for screen reader users to navigate from the error messages area at the top of the form to the field containing the error.
	º	Instruction texts were visually linked to form fields but not programmatically associated with them, meaning that screen reader users would miss this information.
	º	Some fields were marked as “Required” with a red asterisk, but screen reader users might not know that the field was required.

•	On some job application pages, the progress bar was incorrectly defined as a table, meaning that the table structure was announced by a screen reader. In such cases, the progress bar provided only a visual indication of the current step and no programmatic indication.
•	PDF documents found on some pages contained accessibility issues that could prevent screen reader users from perceiving and interacting with their content.

Issues found on mobile versions of web pages:

•	Some pages were not responsive, forcing users to scroll the page horizontally to read and interact with content.
•	While the user menu at the top of some job search pages expanded correctly, VoiceOver users on iOS/Safari were unable to navigate the expanded menu using gestures or touch, thereby preventing them from accessing their profile and account details.
•	When navigating with gestures, the name of the button and menu item were read separately by screen readers, meaning that the name was announced twice.

Summary of user testing findings

The most significant issues identified during the tests and requiring remediation were as follows:

•	When navigating from a non-English version of a page to a job search page that was only in English, the page language was automatically changed to English without warning.
•	In some language selection drop-down lists, the option for English was the only one that was described, while others options were not labelled in a way that indicated to which language they referred (e.g. “3 in 7”, “4 in 7” or “6 in 7”).
•	Some buttons were not clearly labelled.
•	Some images either lacked an alternative description or were not marked as decorative.
•	Not all images containing links (and therefore being “clickable”) clearly indicated where they would take the user.
•	Poor colour contrast between text and background.
•	Some pages (such as job search pages, application forms and new user forms) did not resize to the screen of a smartphone, thereby requiring users to scroll vertically and horizontally to navigate the page. As a result, filling in or editing fields was more difficult.
•	The font size of some text and form elements was very small.
•	Some mandatory form fields were not marked as such, and users were only informed after they had saved the form. All such fields should be clearly marked as mandatory.
•	Some form fields did not provide error messages readable by screen readers whenever the user made a mistake while entering data. The user was therefore forced to wait until after saving the form to find out if there were any errors.
•	The method of indicating errors on forms was frustrating, as it did not indicate exactly where each error was and forced screen reader users to navigate through the whole form again to find the error. This made the process extremely difficult, time-consuming and frustrating for screen reader users.
•	Job application forms in which users are required to fill out many fields did not clearly indicate this at the top of the page (e.g. “In this form, there are 40 fields to fill out”).
•	Job search pages did not resize to the screen size of a smartphone. As a result, users were required to scroll vertically and horizontally to view information and fill or edit fields. This makes filling out a form from a mobile device an almost impossible task.
•	Videos lacked captioning.
•	The job applicant manual was too long.
•	Some job descriptions were extremely long, not correctly formatted and used jargon only known to United Nations employees.
•	Some pop-up windows opened unexpectedly and without notice, which could confuse users of screen readers or force them to lose their navigation focus.
•	Some radio buttons used to provide Yes/No answers were not correctly aligned to the specific questions to which they corresponded owing to poor formatting.
•	The meaning of the options in some drop-down form fields was confusing.
•	Some calendars were exceedingly difficult to use.
•	The “Language page” selection drop-down menu item was labelled as English, when it should be labelled as “Languages”.
•	Screen reader users could not move directly to the list of results of a job search and instead had to tab through a lengthy list of page elements before reaching the list, which was extremely frustrating and time-consuming.
•	Some address form fields were laid out visually in two columns but were not semantically formatted as such. This made it confusing for screen reader users to navigate the form, as the focus would jump from a field on one column to a field in another column.
•	On some pages, the focus of form controls fell on “Save and Continue” as soon as the user landed on the page; this should be the last place where the focus falls, however.
•	Date form fields did not clearly indicate the format to be used and whether to include certain characters between data elements (e.g. a hyphen or forward slash between the day, month and year).
•	There were issues with page language selection management.
•	Some form fields required users to repeatedly select a “Does not apply” option or requested the same information more than once.
•	Some form elements were wrongly labelled as buttons.
•	Different font types and sizes were used, which could be confusing for users.
•	Maps showing the number and geographic location of job vacancies could not be navigated via keyboard. An alternative way of displaying the information is required, such as a list of elements under the map itself.
•	Some buttons were not labelled as form controls, which made navigation more complicated for users of screen readers, given that screen readers prefer to pull up the list of form controls in a page.
•	The heading level order was wrong on some pages.

Recommendations

This report does not pretend to provide legal counsel of any kind. The issues uncovered in the technical and usability report should be addressed aggressively by the teams in each agency. The list of recommendations below should be seen as a call to action to address priority issues, which could significantly improve the accessibility and usability levels of the job portals analyzed:

•	Add alternative text to all images that convey relevant information or content.
•	Use headings to structure page content into clear and descriptive ordered sections in which users can easily find the information that they are looking for.
•	Do not use colour as the only means of conveying specific information.
•	Make sure that the contrast ratio between the text colour and the background colours is sufficient (at least 4.5:1).
•	Apply form labels to all search fields.
•	Use ordered lists for navigation.
•	Avoid using large blocks of justified text.
•	Use autocomplete definitions for input fields so that their purpose is made clear to assistive technologies. It is easier for users of assistive technologies to fill out fields when they know the data type expected.
•	Create PDF documents from accessible source documents, and make sure that adequate semantics/tagging is applied:

	º	Add a proper title.
	º	Add meaningful descriptive text to non-decorative images.
	º	Establish relationships between table headings and data cells.

•	Provide videos with accessible names that can be recognized by assistive technologies.
•	Provide in-page navigation by adding a skip link to allow users to skip over a block of links or text to reach the main content area. Failure to do so makes page navigation more difficult for keyboard-only users.
•	Provide meaningful HTML page titles.
•	Avoid using generic link texts (e.g. “Click here”) as they do not transmit clear information to assistive technology users.
•	Avoid allowing empty links to receive keyboard focus.
•	Place links far enough apart to prevent users from clicking on the wrong link.
•	Avoid structuring page content as tables.
•	Make sure active elements are given a visible focus indicator.
•	Keyboard access:

	º	Provide job search page filter elements to indicate to keyboard-only users whether they can collapse or expand the filter.
	º	Make sure that all video player controls can receive focus and that it is possible to navigate, activate and deactivate the controls using a keyboard.
	º	Avoid using expandable form field options that can only be selected by clicking on them with the pointer.

•	Always define the page language.
•	Where a page includes a language selection button, make sure that the button can be read by screen reading software.
•	Address handling errors in job application or new user forms:

	º	Do not mark required fields with a visible element only (e.g. a coloured asterisk). Required fields should also be marked programmatically so that assistive technologies can recognize that the fields are mandatory.
	º	Include error messages at the top of the form under a heading that identifies the content following it as an error message.
	º	Make error messages as coherent and brief as possible.
	º	Make sure that error messages can be read aloud by screen reading software on computers and mobile devices.
	º	Provide mechanisms for screen reading software to navigate from the error messages area at the top of the form to the field containing the error.
	º	Make sure that instruction texts are linked to form fields programmatically as well as visually so that screen reading software does not miss the information.

•	Provide an indication of the progress towards form completion not only visually (e.g. through a coloured progress bar) but also programmatically.
•	Address issues specific to mobile devices:

	º	Design pages to be responsive, so that they adapt to the size of a mobile device screen and so that users are not forced to scroll the page horizontally to read and interact with content.
	º	Make sure that expanded menus at the top of some job search pages can be navigated using VoiceOver on Safari (iOS) and touch gestures so that users can access their profile and account details.
	º	Make sure that button names and menu items are correctly labelled programmatically so that they are not read separately by screen reading software when navigating through gestures.

Conclusion

All the job portals analysed presented accessibility and usability issues that would pose serious challenges for persons with disabilities interested in learning about and applying to job vacancies with the agencies to which the portals belonged.

Many of the most basic issues, such as the provision of alternative text descriptions for visual content, can be easily fixed. Many other issues are straightforward to resolve and should form part of any short-term web accessibility strategy. Other issues may require a broader and deeper understanding of accessible web development techniques and should therefore form part of a medium- to long-term strategy.

The simple truth is that many leading websites around the world are not accessible to persons with many forms of disabilities and require considerable work to attain the minimum accessibility levels as per international standards. It is imperative that each agency takes the initiative and invests effort into ensuring that their job portals can be used by a broader audience. Greater awareness and increased web accessibility education may also help ensure that persons with disabilities are able to use such portals.

Given the characteristic issues identified across all the job portals analysed, this report may provide a guide for other United Nations agencies on increasing the accessibility of their job portals.