Audit of Information Management Report

Table of Contents

1. Background
   1. 1.1 Audit Objective and Scope
   2. 1.2 Approach
   3. 1.3 Overview of IM within IRB
2. Findings and Recommendations
   1. 2.1 Audit Conclusion
   2. 2.2 Identified Strengths
   3. 2.3 Opportunities for Improvement

1. Appendix A – Risk Definitions
2. Appendix B – Management Action Plan

1. Background

1.1 Audit Objective and Scope

The Immigration and Refugee Board of Canada (IRB) Multi-year Risk-based Audit Plan 2012-13 (RBAP) identified an Audit of Information Management to be conducted during 2013-14.

The overall objective of the audit was to assess the extent to which the Immigration and Refugee Board of Canada's Information Management (IM) framework is effective in achieving the IRB mandate of resolving immigration and refugee cases efficiently, fairly and in accordance with the law.

The sub‑objectives were to assess the extent to which:

• IRB's IM practices meet the business requirements and support decision making across the IRB;
• There is an effective IM governance structure and there are effective supporting tools and service delivery mechanisms in place at the IRB; and,
• IM practices at the IRB comply with the Government of Canada information management framework requirements.

The preliminary audit scope included the governance, risk management and control framework for IM within the IRB, with the following excluded from the audit scope:

• Information Technology (IT); although the IM functionality of IT tools specifically related to records management are in scope;
• Privacy and the protection of personal information;
• Examination of documentation to assess the effectiveness and consistency of application of IRB procedures (unless these procedures are specifically related to IM practices); and,
• Activities and information related to research on human rights and refugee and migration issues.

The audit included a planning phase in order to gain a high-level understanding of information holdings and level of control related to informationwithin IRB. The planning phase included a risk assessment that considered:

• Inherent risk of information holdings based on the inherent likelihood of occurrence and potential severity of impacts on the organization in the event of an information breach or incident (e.g., the confidentiality, integrity, or availability of the information is compromised). Areas with information holdings of higher inherent risk are areas of higher audit interest.
• Extent of current controls related to the information (i.e., how well defined controls are related to the creation/collection, classification, organization, and safeguarding of information). Note that this assessment of current controls was based on the high-level work done during the Planning phase. Areas with more well defined controls are areas of higher audit interest (given audits of areas with a lack of controls do not provide further value beyond reconfirming controls do not exist).

Based on the above assessment, the focus of the audit was further refined to focus on immigration and refugee case files. The IRB's administrative files were not included in the scope of this audit given that:

• the Recordkeeping (RK) Fit-Gap Analysis conducted by IRB from April-June 2013, indicted that the organization has yet to implement a comprehensive IM framework to fully support appropriate record management practices related to IRB's administrative information; and,
• the IRB's critical information holdings are the immigration and refugee case files and not the administrative information.

1.2 Approach

The audit was conducted in accordance with the requirements of the Treasury Board Policy on Internal Audit and followed the Institute of Internal Auditors' Standards for the Professional Practice of Internal Auditing. The audit examined sufficient, relevant evidence and obtained sufficient information to provide a reasonable level of assurance in support of the audit conclusion.

The focus of the audit was on the following areas:

• The IM strategy and action plan (and progress) for the implementation of the IM Framework across the organization;
• IRB's IM governance framework, roles, responsibilities;
• Hardcopy file management process for the four tribunals (i.e., Refugee Protection Division (RPD), Refugee Appeal Division (RAD), Immigration Division (ID), Immigration Appeal Division (RAD)), from receipt/collection to archiving/disposition, including the safeguarding of information;
• How the case management system (i.e., NOVA) is used to track files;
• The registry functions, to understand and test controls for the registries controlled by either the tribunals or the Registry and Regional Support Services (RRSS) Branch (i.e., classification, tracking and management of files, safeguards); and,
• Ownership and management of the relationship with the commercial storage provider.

Site visits were conducted in the Central (Toronto) and Eastern (Montreal) regions. Interviews were conducted by phone with representatives in the Western (Vancouver) Region.

1.3 Overview of IM within IRB

The IRB's Recorded Information Management (RIM) unit resides within the Information Systems Directorate. The Director, Information Systems Directorate has been designated as IRB's Chief Information Officer (CIO). The CIO reports to the Director General, Corporate Services Branch. The RIM unit currently has 8 Full Time Equivalents (FTEs) and has recently reclassified the Chief of RIM position from AS-4 to an AS-6. All RIM resources are located at IRB Headquarters (HQ) in the National Capital Region (NCR).

The information managed by IRB can be profiled as belonging to one of two large categories:

• Related to immigration and refugee case files and decisions; and,
• Related to the administration of IRB.

Immigration and refugee case files and decisions held by the four tribunals are largely in hardcopy format and managed in a structured fashion (i.e. managed by case file, etc.). Three hardcopies registries are used to manage the case files of the four tribunals:

• An integrated registry managed by the RRSS Branch within each of the regions for three of the tribunals: ID (with the exception of the Central Region), IAD, and RAD;
• A separate registry for ID within the Central Region, given this is at a separate location from the main Central Region Office at the Queen's Plate location in Toronto; and,
• A registry for RPD, managed by the division itself.

The divisions use IT systems to support case management (but not the actual case files), specifically NOVA, which is used by the four tribunals to manage cases and track files.

2. Findings and Recommendations

2.1 Audit Conclusion

On an overall basis, the risk exposure faced by the IRB as it relates to IM in the context of the audit is at the high end of the 'Moderate' range (please refer to Appendix A for a definition of the risk rating):

Risk Exposure

Low Risk	Moderate Risk	High Risk

IRB has recently conducted a Recordkeeping (RK) Fit-Gap Analysis, and has begun to address identified gaps through an IM Framework Action Plan. Furthermore, IM has been strengthened through an IM/IT Committee has been actively involved in IM planning and decisions and increasing the capacity of the corporate IM function. Having said that, the audit noted that key foundational items to support IM within IRB have yet to be implemented, including an overall IM Strategy and well-defined and communicated roles and responsibilities that are supported by IM policies, procedures, training, and awareness. These are key items, especially given the complex nature of IRB's organizational structure. Furthermore, the audit noted control gaps related to the tracking and safeguarding of the hardcopy operational case files, which are critical both in terms of their importance to the tribunal process, and the sensitive information which they contain. Given the above, the risk rating for this audit was assessed at the high end of the 'Moderate' range.

The remainder of this document provides additional context and specifics in support of the above summary conclusion.

2.2 Identified Strengths

The audit identified a number of positive practices related to IM throughout IRB. Examples of these practices are listed below:

• A Recordkeeping (RK) Fit-Gap Analysis was conducted by IRB from April-June 2013, and based on identified gaps, 30 different initiatives were identified under an IM Framework Action Plan;
• The IM/IT Committee has been actively involved in IM planning and decisions. IRB's proposed IM Governance structure would help advance the maturity of the organization's IM Framework;
• New positions have recently been created and staffed within RIM, including those related to the development and monitoring of compliance to policy, and for training;
• IRB recently completed a clean-up of case files that were past their retention deadlines;
• Policies and procedures have been developed for the safeguarding of records; and,
• Sound practices related to the safeguarding of records were observed in the Eastern Region (Montreal Office).

2.3 Opportunities for Improvement

To improve IRB's information management practices, the audit has noted seven priority areas that should be addressed on a timely basis. As noted previously, the audit sought to identify the areas of highest importance to the IRB given its current and future environment, and based on information gathered through the audit process, the following seven areas were identified:

1. IM Strategy and Planning (Moderate Risk);
2. IM Governance and Roles (Moderate Risk);
3. IM Policies and Procedures (Moderate Risk);
4. Change Management and Communications (Moderate-High Risk);
5. File Management (High Risk);
6. Safeguarding of Files (High Risk); and,
7. Retention and Disposition (Moderate Risk).

The remainder of this report provides additional details on each of the seven areas.

2.3.1 Finding 1 - IM Strategy and Planning Moderate Risk

Related to IM strategy and planning, the audit expected to find that an overall IM strategy has been implemented, which articulates the vision and mandate for IM within IRB, and is aligned to organizational priorities and key IM risks.

A Recordkeeping (RK) Fit-Gap Analysis was conducted by IRB from April-June 2013, and based on identified gaps, 30 different initiatives were identified under an IM Framework Action Plan. Priority initiatives that were approved and originally scheduled to be completed by the third and fourth quarter of 2013‑14 were:

• IM Governance;
• Identification of Repositories;
• Identification of Information Resources of Business Value (IRBV)^{Note 1}, and;
• Investigation of an Electronic Document and Records Management System (EDRMS).

The completion of some of these initiatives, including the approval of an IM Governance structure, and the identification of IRBV, have now been projected by Information Systems Directorate (ISD) to slip into the first quarter of 2014‑15.

Although the IM Framework Action Plan outlines the key initiatives required to address the identified gaps within IRB's current IM Framework, it does explicitly not outline the vision and mandate for IM within IRB, or place the completion of the outlined initiatives in the context of a "roadmap" on how the initiatives will further move IRB towards this vision. Further to this, the IM Framework Action Plan does not include an overall resource and funding model, and critical path and dependencies for the completion of the initiatives which considers how the initiatives mitigate key identified risks – for example, compliance with the Treasury Board Secretariat (TBS) Directive on Recordkeeping^{Note 2}, which is required by March 2015.

Without a well‑defined IM strategy, there is a risk that individual IM initiatives will not be viewed, actioned, or monitored within the context of achieving the ultimate vision for IM for the organization. 

Recommendations

1. It is recommended that the Chief Information Officer develop an overall IM Strategy to direct the prioritization and alignment of specific IM initiatives. The Strategy should consider the overall vision and mandate of IM within IRB, including considerations for topics such as managing all of the organization's IRBVs in an electronic format. Progress against the IM strategy should be measured periodically to demonstrate to senior management the value of the IM initiatives that have been implemented.

2. It is recommended that the Chief Information Officer ensure that as part of the IM Strategy, the IM Framework Action Plan be supported by a more detailed resource and funding model, and a clearly defined critical path and dependencies for each of the overall Action Plan's initiatives.

2.3.2 Finding 2 - IM Governance and Roles Moderate Risk

Related to IM governance, the audit expected that an enterprise-wide decision-making and accountability framework for IM has been implemented, which includes defined, clear and well communicated roles and responsibilities.

The profile of the Recorded Information Management (RIM) function within the regions is low, as RIM has not formally defined and communicated the services it provides to the organization. It was noted that RIM has had capacity challenges that have curtailed its outreach activities, although it has recently added new positions related to the development and monitoring of compliance to policy, and for training. There are no IM specific positions in the regions, or staff in the regions that report to the corporate RIM function. Despite this, there are a variety of staff members with IM responsibilities in the regions, most notably Records and Mail within Common Services, as well as those with Registry responsibilities. These individuals undertake activities such as file creation, tracking, and archiving. The linkages between RIM and those performing IM‑related activities in the regions are currently not defined, as RIM currently does not provide oversight or direction to the regions on issues such as the retention and disposition of case files. The monitoring of adherence to appropriate IM practices throughout IRB is also not being conducted. Of note, the proposed new governance structure for IM includes IM working groups at the functional level that will be responsible for developing, recommending and integrating IM strategies and implementing IM initiatives, and can be leveraged to strengthen the linkages between RIM and regions.

The result of not having well‑defined roles and responsibilities for IM is an increased risk of confusion on accountability for certain IM activities, and a lack of oversight on overall organization IM practices, increasing the risk of the inappropriate handling of information.

A formal data management and data quality program, including roles and responsibilities has also not been implemented. In the context of the scope of the audit, this was noted in relation to the use of NOVA by the tribunals for case management – in the absence of appropriate data governance mechanisms, there is inconsistency in relation to data entry and data definitions, leading to a higher risk of ineffective and inefficient reporting from NOVA in support of management decision‑making.

Recommendations

3. It is recommended that the Chief Information Officer develop a clearly defined set of services offered (i.e. service catalog) for RIM. This should include RIM's role in the monitoring of adherence to appropriate IM practices throughout IRB. Once the services provided by RIM are defined, key performance indicators (KPIs) should eventually be established to assess RIM's success in the delivery of those services.

4. It is recommended that the Chief Information Officer ensures IRB senior management review and approve an appropriate IM Governance structure, and ensures key linkages are developed between RIM and the regions through the use of the approved governance mechanisms.

5. It is recommended that the Chief Information Officer and the Director General, Policy, Planning and Research Branch establish clear roles and responsibilities for IM and data quality / management throughout the organization.

2.3.3 Finding 3 - IM Policies and Procedures Moderate Risk

Related to IM policies and guidance, the audit expected that a policy framework has been implemented based on legislative and policy requirements, which is communicated, and reviewed regularly.

IRB has not developed an IM policy framework and associated procedures. In the context of the federal government, the overall IM policy and requirements for an organization such as IRB and its staff are set at a high‑level through legislative and policy instruments, most notably the Library and Archives of Canada Act and TBS policy and directives. Despite this, an IRB specific IM policy framework is important to further define and support IM roles and responsibilities within the organization (as outlined in Finding 2), and as a framework for more organizational‑specific procedures which are required to ensure consistent and appropriate application of federal government IM requirements to the specific operational needs of IRB. Of note, RIM has recently added a new position related to the development and monitoring of compliance to policy.

It was also noted that some of the Case Management Manuals (i.e., ID and IAD) used to support the tribunal processes are out of date as they still include reference to processes such as data entry into the System for Tracking Appellants and Refugees (STAR), which has now been replaced by NOVA.

Recommendations

6. It is recommended that the Chief Information Officer develop corporate IM policies and procedures based on legislative and policy requirements that support and address issues such as: records retention, disposition, content ownership in the information lifecycle; identification of managed and unmanaged content; and enforcement of the IM governance framework (as outlined in finding 2). The policy framework should align with the overall IM strategy for IRB (as outlined in finding 1).

7. It is recommended that the Director General, Policy, Planning and Research Branch, update the Case Management Manuals in use within IRB as required, collaborating with RIM as appropriate to determine where linkages to IM procedures are required in the manuals.

2.3.4 Finding 4 - Change Management and Communications Moderate-High Risk

In the context of change management and communications, the audit expected that the culture of the Corporation has been considered throughout the development of the IM strategy and development of practices, a change management and communications strategy has been developed to support the implementation of appropriate IM practices, and a process has been implemented to ensure all staff receive appropriate IM training based on their job position.

The IM Framework Action Plan has identified training and awareness and IM communications and change management as gaps to be addressed through specific initiative; however they have not been prioritized and have not begun. An overall IM training strategy or plan currently does not exist, nor has standard training and awareness content been developed. RIM has not formally leveraged current IM initiatives that involve engagement with the business, such as the identification of IRBVs and repositories, as training and awareness opportunities. Of note, RIM has recently added a new position related to training, although the individual's time is also being used to support IT training. The proposed new governance structure for IM includes IM working groups at the functional level that will be responsible for developing, recommending and integrating IM strategies and implementing IM initiatives, and those individuals within the governance structure can be leveraged to act as IM champions and assist in training and awareness activities. 

In addition to the above, although informal processes exist, there is no formal documented process related to the identification of information owners or the transfer of information from one owner to another owner (e.g. should an owner leave the organization or move to a different position).

Without a change management, training and communications strategy, there is a risk that the IRB will not be able to successfully develop and execute an appropriate IM Framework and program. In addition, by not having a formalized knowledge transfer process in place, there is increased risk that the IRB is losing valuable information that could contribute to the IRB's corporate memory.

Recommendation

8. It is recommended that the Chief Information Officer prioritize change management, communications, and awareness planning and execution activitiesgiven that they are foundational to the development and execution of an appropriate IM Framework and program, and in order to leverage existing work being done with the business (including IM Framework Action Plan initiatives, the finalization of the IM governance structure, and the Email Transformation Initiative (ETI)).

2.3.5 Finding 5 - File Management High Risk

Related to file management, the audit expected that robust controls have been implemented related to the tracking and management of the hardcopy operational case files.

The operational case files for the four tribunals remain as hardcopy records and their movement throughout IRB is intended to be controlled through a sign in/out process using NOVA, which is used by all the tribunals as their case management system. Through audit testing, almost one quarter of the files tested (13 out of 53) were either not signed out to the correct individual, or could not easily be found based on the information provided in NOVA. These file management gaps were noted through audit testing for each of the four divisions (i.e., RPD, RAD, ID, IAD).

Issues were noted in the sign in/out process related to people, technology and process. In some cases it was noted that individuals neglected to sign in/out a file, either because they had forgot or because they had indicated they intended to have the file for only a brief amount of time, but had neglected to return the file to where they had originally retrieved it. Specific to the functionality of NOVA, the sign in/out process in NOVA utilizes an open text box that allows an individual to enter any text, resulting in inconsistent entries or nonstandard locations. Although bar codes have been included on all case files, the use of bar code readers for the signing in and out of files is limited to bulk entries, usually restricted to activities within Records and Mail related to archiving. It was also noted that a formal process does not exist to conduct spot checks/audits, or review NOVA entries and conduct follow-up on case files that have been checked out for an extended period of time, or for issue tracking/analysis when files have gone missing. This type of analysis could be useful to identify "lessons learned" and improve go‑forward practices.

It was also noted that hardcopy case files may contain duplicate documents, for example when the same document is faxed as well as mailed, and in the absence of guidance on this issue, all duplicates are maintained on the file.

Archiving and disposition of operational case files is managed through the Records and Mail function of Common Services within each region separately, including the relationship with Recall (the third party service provider used for offsite archiving). In the Central Region, archiving information (Recall box number, destruction date, etc.) is entered into NOVA, and may also be tracked in an Excel spreadsheet, although it was noted the spreadsheet is not updated consistently. This makes it difficult to understand which files have been sent to Recall without accessing each individual case file in NOVA. In the Eastern Region, an Excel spreadsheet is used to capture archiving information, but this information is not entered into NOVA. There is a risk that, if something were to happen to the Excel spreadsheet, it would be difficult to efficiently determine which files have been sent to Recall. Current practices also make it difficult to reconcile and forecast the total volume of files that have been sent and will be stored by Recall.

Finally, it was noted that although some case files which have been transferred to Recall have reached their 10‑year disposition date, these case files have yet to be disposed, as staff indicated a formal process has not been defined and there is uncertainty as to who ultimately can approve the disposition.

Recommendations

9. It is recommended that the Director General, Registry and Regional Support Services, and the Director General, Corporate Services Branch determine and implement enhancements to the file management process to ensure accuracy of file location and consistency of file treatment (considering the findings noted above).

10. It is recommended that the Chief Information Officer develop a policy and procedures for the retention and disposition of records (as part of the overall IM Policy framework as outlined in recommendation 6).

2.3.6 Finding 6 - Safeguarding of Files High Risk

Related to the safeguarding of files, the audit expected that the safeguards implemented to protect hardcopy files are commensurate with their sensitivity, and Government of Canada (GC) requirements.

IRB Corporate Security has developed corporate security policies and training material, including the Classification and Protection of Information Policy and Procedures,which outlines the appropriate safeguards for protected and classified information consistent with TBS and Royal Canadian Mounted Police (RCMP) requirements. IRB has indicated that operational case files have been determined to be Protected B^{Note 3}. Weaknesses related to the safeguarding of files were noted during the site visits conducted for the audit. Some file rooms (IAD and RAD within the Central Region) that are not staffed are open to all personnel that have an IRB access card, and have no further security protocols. In addition, instances were noted where case files were not safeguarded in accordance with Protected B requirements^{Note 4} when outside of IRB's operational zone. In the Central Region, open mail carts are used to move case files within public areas between operational areas within the same building. Staff members were also noted to carry files in an unprotected (i.e., non-compliant) manner in these same public areas. In the Eastern Region, staff are provided sealed pouches in which to carry case files.

Also, it was noted that filing cabinets approved for the storage of protected information have been provided to the regions but are not always utilized, or when utilized, may not be closed or locked after hours.

Based on interviews conducted, corporate security indicated that similar findings have been identified through Threat and Risk Assessments that have been previously conducted. Through inquiry, it appears that good practices are utilized for individuals who may take case files home, but there is no evidence of a formal policy and/or procedures, or monitoring of adherence to appropriate practices when files are taken home by IRB staff.

Related to the safeguarding of archived files, there was no evidence that assurance has been sought by IRB on the controls used by Recall to safeguard records in Recall's custody. Although it is likely this assurance may have been initially sought by PWGSC, it is important that IRB, as the controlling party of their records, ensure this work has been conducted.

Recommendations

11. It is recommended that the Director General, Registry and Regional Support Services, and the Director General, Corporate Services Branch address the current gaps in the control framework for the storage and transportation of case files (considering the findings noted above).

12. It is recommended that the Director General, Corporate Services Branch and the Director General, Policy, Planning and Research Branch develop a formal corporate policy and procedures related to working on case files at home.

13. It is recommended that the Chief Information Officer, ensure accountability for the relationship with Recall is clarified, which would include ensuring that there is control assurance over its privacy and security controls.

2.3.7 Retention and Disposition Moderate Risk

The audit expected that retention and disposition schedules would be determined for all operational case files based on legal and business requirements.

Case files for RPD, IAD, and ID have an approved Retention Disposition Authority (RDA)^{Note 5} of 10 years, with certain case files for archival or historical purposes retained for 50 years.IRB is currently considering shortening the retention periods of operational case files from 10 to 7 years and from 50 to 18 years. IRB indicates that the 7 year retention period is based on standard administrative requirements and the 18 year retention period is based on discussions with the regions in 2010 which noted that no Access to Information and Privacy (ATIP), Departmental or Inter-Departmental requests for retrieval of records older than 1992 had been made. Of note, a RDA has not yet been approved for RAD, as IRB indicates it prefers to define the new retention periods for operational case files first.

There is an opportunity to conduct additional due diligence on the required retention requirements, including looking at precedents for comparable federal departments, comparable entities in other jurisdictions, considering length of timeline within which appeals can be issued, etc. In addition, having legal analysis and support for this type of decision is also something that is typically sought.

Recommendations

14. It is recommended that the Chief Information Officer conduct additional due diligence on the required retention requirements of case files, including a formal requirements analysis, a review of precedents with similar organizations and jurisdictions, and further legal analysis and support.

15. It is recommended that the Chief Information Officer obtain a RDA from Library and Archives Canada for the new retention requirements, including for RAD case files. 

Appendix A – Risk Definitions

The following definitions were used to assess the risk associated with the observations and overall findings in order to facilitate management's prioritization of action planning and provide an overall rating of the risk exposure surrounding the processes, controls and systems reviewed.

Appendix B – Management Action Plan