Quality Assurance Framework for Decision-Making

​​​​​​​​ ​Printable version (PDF, 609​ KB)

​​​​​​​​​On this page

• 1.0 Introduction to the Immigration and Refugee Board of Canada's Quality Assurance Framework
• 1.1 Overview and objectives
• 1.2 Guiding principles
• 1.3 Approach
• 1.4 Roles and responsibilities
• 2.0 Plan
• 2.1 Identify and prioritize training and mentoring needs
• 2​.2 Identify and prioritize quality evaluation and audit plans
• 2.3 Identify adjudicative strategies
• 3.0 Do
• 3.1 Merit-based recruitment
• 3.2 Training for new members
• 3.3 Professional development
• 3.4 Mentorship
• 3.5 Reasons review by Legal Services
• 3.6 Adjudicative and support tools
• 3.7 Adjudicative consistency groups
• 3.8 Specialized teams/training
• 4.0 Monitor
• 4.1 Performance management
  
• 4.2 Internal and external reporting
• 4.3 Appeal and/or judicial review
• 4.4 Review higher court decisions
• 4.5 Quality centres
• 5.0 Measure
• 5.1 Third-party reviews
• 5.2 Internal evaluations
• 5.3 External audits
• 5.4 Results of member complaints
• 6.0 Adjust
• 6.1 Develop and implement management action plans (audits, third-party reviews)​
• 6.2 Revise training programs
• 6.3 Develop and implement adjudicative strategies
• 6.4 Individual remedial programs
• 7.0 Stakeholder engagement

1.0 Introduction to the Immigration and Refugee Board of Canada's Quality Assurance Framework

1.1 Overview and objectives

The Immigration and Refugee Board of Canada (IRB) has a statutory mandate of resolving immigration and refugee cases efficiently, fairly and in accordance with the law. To ensure that the IRB is well positioned to respond to the changing global migration patterns and unprecedented volume of people seeking access to Canada’s immigration and refugee protection system, the IRB has established a multi‑year Growth and Transformation agenda. The agenda was announced in 2019–20 and is focused on three strategic objectives: improved and sustained productivity; enhanced quality and consistency in decision-making; and strengthened management, with a focus on people management.

As the IRB grows, it is essential that it maintain its reputation for high‑quality and fair decisions. The IRB's Quality Assurance Framework provides an overview of the various activities, processes, strategies and structures that contribute to quality decision-making, as it relates to adjudication, across the IRB’s four divisions.^{Footnote 1} In the judicial setting, courts frequently speak of the need for administrative decision‑making to be transparent, intelligible and justified. These are the elements that mark a quality decision. Consequently, the IRB defines quality decisions as those which can be shown to be transparent, intelligible and justified. The activities outlined in this Framework are therefore closely aligned to these elements and create a path for the IRB to follow towards its strategic objective of enhanced quality and consistency in decision-making. To this end, several principles guide the IRB and decision-makers as they pursue this objective.

1.2 Guiding principles

The following principles guide quality assurance at the IRB:

Compliance: The necessary adherence to legislation, regulations and policies to fulfill the IRB’s mandate.

Consistency: The belief that the decision-making process and the factors that are considered should be applied consistently.

Equity: The belief that all cases should be treated in a fair and impartial manner, in accordance with the law.

Independence: Quality assurance strategies support the accountability of members and facilitate the independence of member’s decision making.

Collaboration: Quality assurance is a shared responsibility across IRB management and members.

Ongoing process of improvement: Quality assurance is not an end result but an integral component of the decision-making processes, with a commitment to the continual improvement of quality assurance strategies and tools.

1.3 Approach

The IRB Quality Assurance Framework is organized according to the stages of a continuous improvement cycle: Plan, Do, Monitor and Measure, and Adjust (see fig. 1). Although many of the activities across the stages run concurrently, there is a natural flow of progression.

Plan: This stage ensures that activities and investments in support of quality decisions are reviewed and prioritized at the onset of the year and revisited at least semi-annually in order to ensure continued alignment with corporate quality and performance objectives. Quality planning focuses on training, mentoring, performance targets and evaluations, which are all aimed at ensuring that best practices are in place for continued improvement. Additionally, updates to adjudicative strategies occur periodically as part of formal legislative reviews and support improved consistency in decision-making.

Do: This stage addresses all the activities which directly support quality decisions in an operational context, where operations at the IRB are comprised of decision‑makers across four divisions: the Refugee Protection Division (RPD), the Refugee Appeal Division (RAD), the Immigration Division (ID) and the Immigration Appeal Division (IAD). Activities include recruiting and training new members, as well as providing ongoing training and mentorship, to ensure continued professional development (PD) for all members. Further focus is put on ensuring that decision‑makers have collaborative forums to engage with their colleagues on current issues and challenges, and that they have the collaborative tools to facilitate the sharing of information.

Monitor and​ ​Measure: This stage is about monitoring and measuring the effectiveness of what was defined in the “Plan” stage and performed in the “Do” stage. It answers questions about the effectiveness of our training and mentorship programs. It reviews the quality of decisions, ensuring we have the right practices in place to support decision-making excellence, including timely and complete pre-proceeding readiness; respectfulness of proceedings; focused proceedings; and clear, complete, concise and timely decisions. The activities at this stage are facilitated both at the division level, where quality centres are being established, and centrally, where independent third‑party reviews are facilitated.

​ Adjust: The final stage of the Framework is comprised of activities to address the changes that need to be either incorporated immediately into operations (the “Do”) or included as input into the next planning cycle. This includes activities such as the development of action plans to address findings from third‑party reviews and external audits, revisions to training programs, and adjustments to adjudicative strategies in response to changing trends. Finally, this stage provides for defining remedial actions to support individual members in their professional development.

Fig. 1: IRB Quality Assurance Framework for Quality Decision-Making

*New initiative to be implemented in FY2020-21​

1.4 Roles and responsibilities

The IRB’s leadership plays an important role by promoting and emphasizing innovation and teamwork, which provides an important foundation for implementing the Quality Assurance Framework. However, lasting results cannot be achieved without the commitment and support of the entire organization. Within this collaboration, several groups play integral roles in upholding quality assurance:

Chairperson: The Chairperson provides overall leadership and direction with respect to quality assurance and professional development.

Deputy chairpersons: The deputy chairpersons are accountable to the Chairperson for the quality decision-making of the members assigned to their divisions. They provide vision and set priorities with regard to quality programs, and enable assistant deputy chairpersons to implement these priorities.

Assistant deputy chairpersons: Assistant deputy chairpersons provide leadership in implementing quality programs and ensuring that appropriate resources are committed to support quality decision-making. They create an environment for their teams that facilitates and fosters continuous improvement and quality decision-making. Representative assistant deputy chairpersons from each division will play an integral role in the management of the quality centres.

Quality centres: The quality centres will provide focused attention on initiatives, tools and mechanisms that improve quality. They will also provide a mechanism to ensure that best practices are communicated and that quality issues are identified and addressed in a timely manner, independent of the individual performance management system.

Coordinating members: Coordinating members support and encourage members’ personal development by providing regular feedback and identifying areas for development with respect to quality decision-making. They provide advice on procedural and other issues, identify training needs or areas of inconsistency in member performance, and promote the use of best practices.

Members: Members are responsible for appropriately applying legislation, regulations and policies, and for employing strategies and tools to render quality decisions. They also participate in learning and personal development opportunities for the purpose of enhancing competencies, thereby strengthening quality decision-making.

Legal Services: Legal Services prepares jurisprudential updates, case notes and other essential documentation contributing to quality decision-making. Legal Services is available to assist members by addressing legal questions and preparing quality reasons. It is also involved in the training of members.

Strategic Directions and Corporate Affairs Branch​: The Strategic Directions and Corporate Affairs Branch maintains the Quality Assurance Framework. It is responsible for many of the monitoring and measuring activities, and the reporting of results.

2.0 Plan

2.1 Identify and prioritize training and mentoring needs

The identification of training and mentoring needs is an ongoing process for the RPD and the RAD. Training and mentoring needs are identified so that members receive training and mentoring support where it will be most impactful to ensure quality adjudicative decision-making. Within the RPD and the RAD, coordinating members (supervisors) consider member requested needs, data trends, performance information and stakeholder feedback when designing new member training, developing training materials, establishing mentorships and holding professional development (PD​) sessions.

Regionally based PD​ committees establish forward agendas for PD training based on current issues and needs identified in their regions. For example, for the RPD, each PD committee is composed of members and legal advisors, and is chaired by a coordinating member. The committee meets bi-weekly or monthly to identify training needs and to plan PD sessions. The committee is responsive to training needs identified by RPD management and Legal Services for members, and provides the forum for such training in its regularly scheduled PD sessions.  

At the RAD, a national PD training committee sets learning objectives for the year to ensure ongoing PD. Priorities are identified with input from the Adjudicative Strategy and the PD committees, and emerging Federal Court jurisprudence. Other inputs are received from Legal Services and management action plans (MAPs) developed in response to quality measurement initiatives (QMI​s)—the latter is facilitated by the Strategic Planning, Accountability and Reporting (SPAR) Directorate.

Mentors and needs for mentorship are identified within each division of the IRB and take three forms: 1) New member mentorship (peer-to-peer colleague support [“buddies”] and assigned mentors); 2) Ongoing PD (mentorship requested by a member to improve a specific skill); and 3) Remedial/reactive mentorship (based on an identified performance need). With the exception of peer‑to‑peer mentorship, mentors are generally hired on contract from amongst former IRB members.   

The Member Learning Secretariat (MLS) plays a key role in supporting this work. The goal of the MLS is to provide national, organization-wide coordination of learning and PD activities for decision-makers such as new member training, PD sessions, mentoring and national training symposiums. This support comes in the form of planning training and learning activities, preparing training material, coordinating with regional counterparts for venues and equipment, tracking training sessions, etc. The MLS is also building capacity to be able to support the identification of training needs by employing qualitative assessment tools such as surveys.

2.2 Identify and prioritize quality evaluation and audit plans

The planning of the IRB’s third-party reviews and internally led evaluations helps to ensure that all components of quality decision-making are reviewed, and that areas for continuous and ongoing improvement are identified.

The evaluation unit within the Strategic Planning, Accountability and Reporting (SPAR) Directorate conducts an annual planning exercise to identify and prioritize all third-party reviews, as well as internally and externally led evaluations over a five-year period. This process is conducted in accordance with the Treasury Board of Canada Secretariat (TBS​) Policy on Results and is facilitated through the use of the TBS’s Evaluation Coverage Tables.

As part of this exercise, some reviews such as quality measurement initiatives (addressed in Section 5.1 - Third-party reviews) are planned for each division on an annual basis, while other reviews are planned at the request of internal IRB stakeholders, the TBS or Parliament. Reviews that are initiated internally seek to ensure the quality of initiatives that support adjudicative decision-making, as well as the quality of the decisions themselves.

The proposed list of prioritized evaluations and audits are reviewed and approved by the Audit, Evaluation and Performance Measurement Committee (AEPMC).

The IRB rarely initiates or conducts its own audits; rather, the audits are conducted by the Office of the Auditor General (OAG) or the Office of the Comptroller General (OCG​). The coordination of responses to these audits and the development of Management Action Plans (MAPs) is the responsibility of the SPAR Directorate and is integrated into the SPAR Directorate’s plans.  

2.3 Identify adjudicative strategies

At a foundational level, adjudicative strategies guide the IRB as a tribunal and drive consistency and quality in decision-making by ensuring decision‑makers (members) keep the IRB’s “raison d’être” front of mind. While each decision‑maker is independent, adjudicative strategies are used to model best practices, to provide examples of sound decision‑making strategies, to explain complex areas of law and to explore unsettled questions of fact.

The Board’s adjudicative strategy initiatives can be loosely grouped into two categories:

• soft law instruments, for example guidelines, policies and practice notices; and
• adjudicative instruments, such as jurisprudential guides, persuasive decisions, three‑member panel decisions and reasons of interest, which consist of individual decisions that are utilized for strategic adjudicative purposes.

Collectively, these tools are intended to facilitate efficient and fair decision-making that is responsive to developments in both jurisprudence and the evidentiary foundations on which decisions are based.

Adjudicative strategies are identified and implemented in order to enhance quality, consistency and efficiency in decision-making. For example, where the RAD uses one of its adjudicative tools to map out a legally sound framework for analysis that can be used in a large number of cases, this framework can then be used by other decision-makers to render legally sound decisions in an efficient manner. The framework does not dictate a result, or infringe independence, but it does foster a consistent approach to adjudicating similar cases. Decisions with respect to the use of adjudicative tools are driven by a number of factors, including assessments of common case types, unresolved legal and factual issues, and country volumes. The use of these tools is developed in collaboration with adjudicative strategy committees and their respective divisions. Once designated, these tools are evaluated and monitored based on their ongoing utility to members and their continued accuracy and relevance. Examples of existing adjudicative strategies include jurisprudential guides, persuasive decisions and reasons of interest, which are published online.

Planning related to adjudicative strategies occurs on an ongoing basis through the use of metrics to support the forecasting of legal issues or to identify areas of inconsistency, for example. This allows the IRB to map out particular topics and determine if use of an adjudicative tool or strategy may be warranted. Additionally, key legislative changes (including to the Rules) trigger significant planning events to review, improve or adjust our adjudicative strategies. More flexible policy instruments, such as practice notices, are used to make minor adjustments, if required, between major legislative reviews. (More information on this aspect is provided in Section 6.3 – Develop and implement adjudicative strategies.)

3.0 Do

3.1 Merit-based recruitment

The IRB has developed staffing strategies and tools to ensure future decision makers are able to render fair and efficient decisions. During the hiring process, applicants must successfully complete a written exam and an interview that are tailored to measure decision-making ability, alongside other necessary qualities, such as judgment and reasoning, information-seeking and self-control. A reference check and background check are also administered to verify the candidate’s personal suitability for the member position, and a values and ethics assessment is conducted. The staffing strategy is responsive to data trends, such as determining the predicted number of applications needed to fill a set number of positions.

For the RPD, merit-based recruitment is in place to ensure that persons hired are qualified and able to render quality decisions. At the RAD, in collaboration with Governor in Council Secretariat Services, a list of qualified persons is presented to the Minister for appointment by the Governor in Council (GIC).

3.2 Training for new members

Led by experienced member trainers and co-delivered with Legal Services counsel, New Member Training (NMT) combines enhanced pedagogy with teaching to the theory of the case, helping members focus on determinative issues and deliver quality decisions. It is built on the principles of adult education training, and incorporates hands-on member experience through job shadowing and participating on three-member panels. This training runs for approximately seven weeks, and ensures that all new members have a standardized approach to conducting hearings and rendering decisions. NMT is regularly updated to reflect changes in policy, procedures and law. Lessons learned are created at the end of each session to promote the quality and integrity of the training. The training approach was developed with help from a third-party consultant (a professor from Saint Xavier University with expertise in adult learning).

A Member Learning Secretariat (MLS) was created in spring 2019, as a resource for all divisions to support certain facets of the administration of member training. The goal of the MLS​ is to provide national, organization-wide coordination of learning and professional development activities for decision-makers, including NMT. This support comes in the form of planning training and learning activities, preparing training material, coordinating with regional counterparts for venues and equipment, tracking the training sessions, etc.

NMT​ ensures that all new members receive the knowledge, tools and support required to develop their decision-making skills to a level where their decisions are of high quality.

3.3 Professional development

Monthly Professional Development (PD) sessions provide an opportunity for members to reflect on pressing substantive and legal issues, including feedback resulting from RAD decisions and/or judicial reviews of the Federal Court. A typical PD session includes a jurisprudential update by Legal Services on recent case law and a member-led component on a selected topic. These monthly sessions provide continuous learning opportunities for all members on matters most pertinent to their work so that they are well informed about current issues and better equipped to make quality decisions.

For the RPD, currently each region develops its own PD sessions based on emerging issues. In each region, PD sessions are organized by a committee that consists of up to five members, a coordinating member and a Legal Services advisor. The PD committee solicits interest from members about topics on which they would like to have sessions and also generates its own ideas for PD sessions. Additionally, the PD committee is responsive to training needs identified by RPD management and Legal Services for members. In some regions, a reasons writing committee exists that is focused on providing tools and training for members to render concise and timely decisions.

PD at the RAD is guided by a national PD committee, in collaboration with the Deputy Chairperson’s Office. PD sessions are offered nationally in both languages. Additionally, informal regional PD sessions are frequently conducted by regional assistant deputy chairpersons.

PD sessions cover a wide range of topics from jurisprudential and policy updates, to issues of hearing room management, to the finalization of less complex claims, and the efficient rendering of decisions. Additionally, they cover topics such as avoiding burn-out, sensitivity fatigue, resilience and other areas that address the mental health and wellbeing of members. These topics contribute to better working environments, which result in a better quality of work.

3.4 Mentorship

Mentorship aims to ensure that members have access to support structures. In the case of remedial mentorship, it also allows for the targeting of identified performance challenges of specific members. Mentoring support reinforces training material and helps members develop and maintain the skills needed to make high quality decisions.

Mentorship at the IRB takes three forms: 1) New member mentorship (both peer-to-peer colleague support [“buddies”] and assigned mentors); 2) Ongoing professional development (mentorship requested by a member to improve a specific skill); and 3) Remedial/reactive mentorship (based on an identified performance need). With the exception of peer to peer mentorship, mentors are generally hired on contract from amongst former IRB members.

The RPD New Member Buddy Initiative supports new members by informally providing opportunities to discuss case preparation and the decision-making process with experienced members. The initiative allows new members to be exposed to various ways of organizing their workload and practical matters such as managing administrative tasks and procedural issues in the hearing room.

The RPD’s proactive National Mentorship Program helps new members transition into a full hearing schedule and implement lessons learned from training. This provides support for new members during the first four to six weeks of decision-making. It is the next step in adjudicative development after New Member Training and three member panels. The program’s goal is to create and maintain the skills needed to support fair, efficient and high-quality adjudication.

As part of the RPD Member Mentorship Program, RPD mentors address identified areas for improvement in a member’s performance. Mentors closely review files with members, observe hearings and read finalized decisions. RPD mentors can be requested by either the member, or assigned by management after performance evaluations. RPD mentors contribute to the success of new members, as well as the growth of more experienced members, by enhancing the quality of their work through coaching.

The RAD has implemented internal (current members) and external (former members) mentoring programs. Mentorship in the RAD is similar to that in the RPD; it is provided proactively to all new members. It is continued for varying amounts of time depending on the new members’ needs, and it is offered remedially in situations where particular performance needs have been identified.

3.5 Reasons review by Legal Services

Legal Services counsel is available to assist members in preparing quality reasons that accurately reflect the rationale for their decisions and offering advice that adopts a legal risk approach to possible RAD or Federal Court review. Under the IRB’s Reasons Review Policy and Instructions Governing Solicitor-Client Privilege, reasons reviews by counsel are confidential to members, and voluntary, although an exception is made during the training period so that management can be apprised of any training needs. Members are assigned a Legal Services advisor who is available to address any legal question, whether it be case specific or a broader adjudicative/legal question.

While reasons review is generally an optional service offered to decision makers, the training of new members is a special situation, in which legal counsel work with them as part of a team. This team may be composed of a professional development advisor, an experienced member, acting as mentor, and the new member's manager. Therefore, new members are expected to make use of the reasons review service offered by Legal Services in accordance with the New Member Training Program.

Additionally, a member manager (coordinating member) may recommend to a member that they should have Legal Services review their draft reasons because of the complexity or novelty of the case or because the member manager is of the view it would be consistent with the objectives of reasons review.

When providing reasons review, counsel seek to promote quality, efficiency and independence along with consistency in their advice, including in providing timely advice to members seeking to render oral decisions. However, how information is leveraged is at the discretion of the member, as the decision and reasons remain their own. The decision maker is free to accept or to reject comments made by Legal Services on the reasons. Legal Services does not share any specific cases or concerns with management, but may raise frequently noted issues and trends, for purposes of identifying general training needs.

3.6 Adjudicative and support tools

Adjudicative and support tools are developed to facilitate quality decisions by improving procedural regulations, information access, and/or horizontal member engagement. As described in Section 2.3 – Identify adjudicative strategies, these tools, which support adjudicative strategies, lead to improved quality through enhanced consistency in decision-making.

Adjudication tools are primarily written guidance for members—and in some cases the Registry, or others involved in proceedings before the IRB—dedicated to improving the quality of IRB regulations that assist members in achieving their mandate.

Adjudicative tools include:

• The Chairperson’s guidelines, which provide guiding principles for adjudicating and managing cases;
• Jurisprudential guides, which support consistency in adjudicating cases which share essential similarities;
• Policies, which are formal statements that explain the purpose and the mechanics of operational initiatives at the Board; and
• The Chairperson’s instructions, which provide formal direction that obliges specific IRB personnel to take or to avoid specific actions.

Adjudicative tools can also cover specific situations (e.g. Guideline 9: Proceedings before the IRB Involving Sexual Orientation and Gender Identity and Expression or jurisprudential guides).

Members make use of adjudicative tools to reach their own conclusions based on the facts of each particular case. Although not binding, members are expected to follow guidelines, unless compelling or exceptional reasons exist to depart from them.

Support tools are primarily concerned with improving information access and horizontal member engagement, which subsequently enhances decision quality. The Knowledge Management Tool (RPD only), up-to-date country statistics and national documentation packages, and responses to information requests are all tools that support information accessibility and also help managers with monitoring trends. The RAD’s Members-Only Collaborative Workspace is a support tool that improves horizontal engagement of members by enabling online consultation and feedback. The RAD member tool provides an electronic forum for members to exchange information and ask questions about issues of mutual concern.

These tools aim to ensure both the consistency and the quality of decision-making and proceedings more broadly at the Board and allow members to efficiently and consistently consult knowledge that assists them in carrying out their duties.

3.7 Adjudicative consistency groups

RPD members meet in adjudicative consistency groups (ACGs) to discuss adjudicative trends and issues related to emerging country conditions, claim type analysis, case law, etc., in an effort to improve adjudicative quality and consistency in divisional decision-making. This allows a forum for members to discuss and understand trends and issues in depth and in a consistent manner.

ACG meetings are Consolidated-Bathurst type discussions led by a member. These meetings typically last about two hours and occur once a month. They are designed to enhance the consistency and quality of the decision-making objective of the RPD. ACG meetings allow all members adjudicating the same countries to have plenary discussions on issues of prevailing country conditions, policies, procedures and law, while respecting the adjudicative independence of the participating members. Members discuss these issues amongst themselves, share their experiences and ask questions of their colleagues. Highlights from these conversations are captured in summary reports (Feedback Forms) that provide a record of the major issues discussed, the material that was provided, and any follow up that may be required.

The objective of ACG meetings is to ensure that members are fully aware of all the issues, trends and developments in the countries from which they hear claims. ACG meetings are not meant to be prescriptive about what decisions members should make in their hearings. ACG meetings help to address matters of intra-team variance related to both decision-making and the efficient use of hearing room time.

3.8 Specialized teams/training

Specialized teams and task forces have been used for various reasons in the past (to address backlogs or specific regional issues, for example) and are currently being created for quality purposes.

Specialized teams (whether divisional or regional), and the related specialized training, allow the RPD to develop advanced subject matter expertise in areas related to country conditions, claim types, questioning techniques, and dealing with sensitive claims and vulnerable claimants. Specialized teams afford the RPD opportunities to test quality improvement and adjudicative strategy initiatives and to receive timely feedback. New emerging issues can also be identified efficiently.

Country specialization is a foundational principle for the assignment of cases to RPD members in the Central Region. Teams tend to be specialized in a set number of countries, and within each team there can be further specialization, which allows members to develop an in-depth understanding of country conditions and claim types. Members are therefore well placed to recognize patterns and respond appropriately when country conditions quickly change. Country specialization also leads to greater efficiency within the hearing room and in the rendering of decisions.

Country specialization considers organizational needs but also members’ expertise and preferences. Where a limited volume of cases precludes country specialization, countries are assigned based on regional or thematic specialization.

4.0 Monitor

4.1 Performance management

When it comes to managing the performance of employees and members, the RPD follows the Public Service Performance Management (PSPM​) Cycle, as set by the Treasury Board of Canada Secretariat (TBS). This includes setting performance objectives at the beginning of the fiscal year, and conducting mid year and end of year reviews. It also includes an assessment of the member against the four core competencies of the Public Service. In addition, members receive regular reports on their performance in terms of timeliness of decisions and productivity, and meet with their managers on a monthly basis to discuss their performance. The member Code of Conduct is included as a measure in these reviews. In addition, coordinating members (supervisors) will regularly review decisions as part of this cycle and will, on occasion, pull audio files to listen to how hearings are conducted. Any identified issues are addressed on an ad hoc basis, as well as in a more structured fashion during the formal performance cycle.

Supervisors build quality into performance management agreements at the beginning of each year. While there is not uniform language regarding quality, there is a nationally agreed upon approach for performance expectations (120/year, 90% finalized within 25 days, 100% finalized within 50 days).

For new members in the RPD, there is an onboarding process that includes staged assessments. This process usually takes six to eight months. A template is filled in for each new member for each of two observations at each assessment stage, and four decisions are read for each new member at each assessment stage. The completed templates are then consolidated into a single report per assessment stage for each member and then discussed in a committee consisting of all those who are involved in training and onboarding. These meetings permit additional training, coaching or mentoring needs and measures to be identified and implemented. The process is overall robust and has produced good results, and because the program is so comprehensive, should the need arise to end a member’s contract within the probationary period, the RPD would be in a good position to do so.

Given that RAD members are GIC​ appointees, they do not fall under the PSPM; however, the RAD follows a separate, similar formal review cycle, with annual and end of mandate reviews.

The Strategic Human Resources Management (SHRM​) Directorate provides support on a variety of aspects of the organization, including Performance Management (PM) and Talent Management (TM​). The PM​/TM team provides support to the entire IRB, including the RAD and the RPD. It also acts as the link between the IRB and the TBS, which requires reporting on PM and TM through the Management Accountability Framework.

The numerous means available to assess, motivate and manage IRB members ensure that decisions, reasons and hearing room conduct are of high quality.

4.2 Internal and external reporting

Regular monitoring of the IRB’s progress against priorities, targets and other key activities ensures the IRB is able to recognise challenges and continually make adjustments in order to meet our quality objectives. Regular reporting and publishing of these reports also ensures a level of transparency and accountability with respect to the IRB’s activities, including results, audits, etc.

Internal reporting related to quality is coordinated by the Strategic Planning, Accountability and Reporting (SPAR) Directorate through regular updates to the Audit, Evaluation and Performance Measurement Committee and the Executive Management Board. These executive level committees review results of audits and evaluations, as well as updates on priority initiatives, including quality. Additionally, the Standards, Analysis and Monitoring (SAM) unit produces regular reports that provide information on divisional performance, including quality.

External reporting related to quality is also coordinated by the SPAR Directorate through public reports to Parliament such as the Departmental Plan (DP) and the Departmental Results Report (DRR), which are submitted to Parliament and published on the IRB’s external website. These public reports indicate plans and results related to the program targets of the IRB, one of which is quality of decision-making. The reports indicate the planned and actual percentage of cases that meet quality standards.

The results that are indicated in the DRR​ are based on the IRB’s quality measurement initiative (QMI) whereby a third-party reviewer assesses a sample of decisions from each of the four divisions (RPD, RAD, ID​ and IAD) against a checklist of procedures that ensure quality decision-making. More information about QMIs is available in Section 5.1 – Third-party reviews.

Regular reporting helps to determine the IRB’s overall results for decision-making excellence, including: timely and complete pre-proceeding readiness; respectful proceedings; focused proceedings; and clear, complete, concise and timely decisions.

4.3 Appeal and/or judicial review

An appeal to the RAD or a judicial review, with leave, at the Federal Court are legal remedies where eligible parties to the initial decision think an error has been made and/or believe that a breach of natural justice has occurred. These function as corrective measures and may also be seen as qualified indicators of quality decision-making when decisions at either division are upheld.

The RAD reviews decisions of the RPD when the claimant or the Minister believes that an error was committed by the RPD or that a breach of natural justice occurred, or wants the RAD to review the RPD decision in light of information that could not have been presented to the RPD prior to the RPD decision.

The RAD decides to either:

• confirm the determination of the RPD;
• set aside the determination and substitute a determination that, in its opinion, should have been made; or
• refer the matter to the RPD for re-determination, giving the directions to the RPD that it considers appropriate.

When the Federal Court decides to judicially review the reasons for the refugee appeal decision, or in limited circumstances, the refugee protection claim decision, it will decide whether the decision was reasonable. A hearing where both the claimant or appellant and the Minister’s counsel provide arguments is part of this process. The Federal Court may grant or dismiss the judicial review. If it is granted, the matter is returned to the IRB for re-determination.

The IRB may decide to seek leave to intervene in a higher court proceeding to explain to the court the Board’s adjudicative strategy, to make submissions on jurisdiction or to explain operational considerations. In any case in which the IRB seeks leave to intervene, the impartiality of the Board must be maintained.

Summaries of RAD and Federal Court decisions are regularly shared with members by Legal Services. These summaries allow members to be updated on how the appellate bodies view the findings of the RPD. Any decisions of the RAD and the Federal Court that are of particular significance to the RPD are discussed in greater detail at Professional Development sessions.

4.4 Review higher court decisions

Federal Court decisions are reviewed by RPD and RAD management and Legal Services to assess whether they identify particular Professional Development needs or member-specific concerns.

Legal Services provides jurisprudential updates on Federal Court and RAD decisions as well as develops specific training in light of new or novel jurisprudence.

A Higher Court Decisions database is maintained with case notes of decisions dealing with refugee and immigration matters. Most of the case notes are from decisions rendered by the Federal Courts of Canada. However, it also contains selected decisions of other courts (e.g. the Supreme Court of Canada and provincial courts). The application includes case notes of decisions rendered since July 22, 1996. The case notes are prepared by the IRB’s Legal Services. Each case note includes a string of keywords and a hypertext link that leads to the full-text reasons, when available.

4.5 Quality centres

The quality centres in the RPD and the RAD are being established in fiscal year 2020–21 to monitor the quality of decision-making. When fully implemented, quality centre data analysts will mine RAD and RPD decisions, as well as Federal Court cases in order to identify specific legal questions that repeatedly occur. The Adjudications Operations Committee will meet regularly to discuss quality centres and adjudicative strategies to address these issues. Joint RPD/RAD meetings will also be held to discuss legal issues and approaches to questions concerning refugee determination.

The quality centres will provide focused attention on initiatives, tools and mechanisms that improve quality, such as improved training, mentorship, performance management and adjudicative strategies. They will ensure that legal issues in decisions are identified early on, and appropriate adjudicative strategies are adopted in response. They will provide a mechanism to ensure that best practices are communicated and that quality issues are identified and addressed in a timely manner, independent of the individual performance management system. For example, if a quality centre noticed a trend, such as an increase in the postponement rate due to council not being able to access certain documentation, it would trigger a discussion on how the issue could be addressed, with the goal of encouraging consistency and adherence to rules. In this case, a new adjudicative strategy might be proposed, such as the issuance of a practice notice.

5.0 Measure

5.1 Third-party reviews

Third-party reviews refer to quality measurement initiatives (QMIs) and external evaluations. In these cases, a third-party assessor is recruited to maintain objectivity and conduct reviews on the quality of initiatives and decision-making. Third-party reviews of IRB programs, processes and procedures help to identify strengths and areas for improvement by measuring the quality of initiatives that support decision-making, and also help to measure the quality of decision-making itself.

The Strategic Planning, Accountability and Reporting (SPAR) Directorate leads annual QMIs, which assess the quality of decision-making within each IRB division, through the use of a third-party assessor who follows a standardized approach. The assessments are conducted on a sample of cases for a given division and are measured against a set of standards for quality decision-making. Within each QMI, Gender-Based Analysis Plus (GBA+) is also conducted, as per guidelines by the Treasury Board of Canada Secretariat which indicate that GBA+ must be conducted for all new initiatives and evaluations. Strengths and areas for improvement are identified in the final report and recommendations are provided. The divisions use the findings to support improvements to their programs and, more specifically, to the quality of member procedures and decisions making. External evaluations and other reviews are often initiated as a result of findings from other processes (complaints, QMIs or other external influences).

A report with recommendations is produced following all third-party reviews. The IRB responds with a Management Action Plan (MAP), which is monitored by the SPAR Directorate and updated by the office of primary interest. Reports and MAPs are presented and monitored through the Audit, Evaluation and Performance Measurement Committee (AEPMC), which is chaired by the IRB Chairperson. Examples of MAPs related to quality assurance include: improved support for the planning, design and delivery of member training programs; and provision of professional development refreshers regarding member questioning.

Third-party reviews are a necessary component of assuring quality within the IRB. While audits function to assess the priorities of external stakeholders, third-party reviews function to assess the priorities of internal stakeholders. Third-party reviews are coordinated internally, but the evaluative component is outsourced to ensure that final reports remain objective.

5.2 Internal evaluations

An evaluation in the Government of Canada is a systematic and neutral collection and analysis of evidence to judge merit, worth or value. It determines the extent to which a program or project has achieved expected results. Evaluation informs decision-making, improvements, innovation and accountability.

The Evaluation Unit within the Strategic Planning, Accountability and Reporting (SPAR) Directorate supports the IRB divisions or branches in assessing the quality of new and ongoing initiatives/programs. The SPAR Directorate facilitates the development of logic models and theory of change models against which success can be measured. In a leading role, and in consultation with the Standards, Analysis and Monitoring (SAM) unit, the SPAR Directorate supports the development of performance measures and methodologies for collecting data to inform performance. The SPAR Directorate analyzes the results, and reports findings to the client.

Conducting evaluations improves IRB decision quality by assessing the outcomes of new initiatives that support decision-making. For example, the SPAR Directorate is currently working on two evaluations for a new position within the RPD and the RAD respectively. Each of these divisions has created a new position which seeks to support members by ensuring that case files are hearing ready, and that members have accurate and reliable information about every case, each with the end goal of fair and timely adjudication of cases. The evaluations will help determine whether these positions are successful in meeting this objective and will help inform investment decisions about these new positions going forward.

5.3 External audits

Auditing is a professional, independent and objective assurance and consulting activity designed to add value and improve an organization's operations. It helps an organization accomplish its objectives by bringing a systematic, disciplined approach to assessing and improving the effectiveness of risk management, control and governance processes.

External audits of the IRB are typically conducted by the Office of the Auditor General (OAG) and the Office of the Comptroller General (OCG) which send out routine and non routine call letters to inform departments of new audits and the associated areas that will be assessed. The role of the IRB, with the Strategic Planning, Accountability and Reporting (SPAR) Directorate as the lead, is to analyze the audit request, coordinate responses with offices of primary interest, and compile responses and evidence for submission to the office that is conducting the audit.

Following the OAG/OCG’s examination, a draft report is shared with the IRB to confirm the accuracy of their findings, review recommendations, and when necessary, respond to recommendations with Management Action Plans (MAPs). Reports and summaries are published on the IRB website. The OAG/OCG monitor these MAPs and request annual or semi-annual updates. Once artefacts are provided that demonstrate that the IRB has fulfilled the audit recommendations, MAP items can be closed.

Independent, objective and systematic audits of the IRB represent a vital resource for maintaining and improving decision quality at the IRB. It presents an opportunity for the IRB to identify program gaps and initiate departmental improvements.

5.4 Results of member complaints

Anyone may file a complaint about the conduct of an IRB member. The complaint procedure is meant to address a members’ professional conduct. It provides a formal mechanism for counsel, any party to IRB proceedings, IRB personnel, an organization or the public to raise any challenges in member behaviour that they see as counter to the IRB Member Code of Conduct. Complaints must be made in writing to the Director of the Office of Integrity. A complaint form and procedures guide are available on the IRB public website.

Complaints against members that fall under the scope of the Member Code of Conduct are examined by the Office of Integrity, or in exceptional circumstances, an external investigator. A report of the investigation is provided to the Chairperson, who will examine it and decide whether to accept the conclusions, and communicate the outcome to the complainant, the member concerned, and the Assistant Deputy Chairperson and Deputy Chairperson. The Chairperson will decide whether any follow up actions are required. Overall system trends are monitored and used to inform updates to training programs as required.

Aggregate information regarding numbers and disposition of complaints against members is published on the IRB’s website. Publishing aggregate information regarding results of member complaints on the website creates transparency.

6.0 Adjust

6​.1 Develop and implement management action plans (audits, third-party reviews)​

Management Action Plans (MAPs) or Management Response and Action Plans (MRAPs) are developed in response to recommendations indicated in audits, and third-party reviews such as quality measurement initiatives (QMIs) and external evaluations. The Strategic Planning, Accountability and Reporting (SPAR) Directorate coordinates the development of a MAP/MRAP by engaging the office of primary interest (typically a division) to review the recommendations of an audit or third-party review. The office of primary interest will then respond by accepting or rejecting it, and providing a plan for how they will address the recommendation, including timelines. MAPs and MRAP​s are presented at the Audit, Evaluation and Performance Measurement Committee (AEPMC) for approval. Progress against the MAP is monitored and tracked by the SPAR​ Directorate, and brought back to AEPMC to ensure ongoing accountability of implementation.

MAPs are integral to quality improvement as they consider identified program or process gaps and specify next steps for resolving them, leading to improvements in processes, training and results. Beyond outlining a plan of action, the development of a formal MAP ensures ongoing accountability to both internal and external stakeholders. MAPs remain active until sufficient evidence is provided to ensure gaps have been addressed and functional quality is improved.

6.2 Revise training programs

Member training programs for new and experienced members are reviewed on a continuous basis by the divisions, in consultation with and with the support of Legal Services. Training revisions ensure that training methodology and content are up to date with best practices, and allow new tools, strategies and research to be integrated into the training program. Additionally, revisions consider whether any changes in law, findings through QMIs, or systemic issues identified through performance management and the member complaint process warrant updates to the training program.

For the RAD, this is an “evergreen” activity performed in the period preceding scheduled new member training (NMT). At a minimum, relevant jurisprudence that has emerged since the last delivery of NMT is updated. Other revisions will be made subsequent to discussions of the Adjudicative Strategy and Professional Development committees, or upon the recommendations of the lead of the Quality Centre.

6.3 Develop and implement adjudicative strategies

As identified in Section 2.0​ – Plan, adjudicative strategies are central to the IRB’s efforts to enhance both consistency and quality in decision-making. The Board’s adjudicative strategy initiatives can be loosely grouped into two categories:

• soft law instruments, for example guidelines, policies and practice notices; and
• adjudicative instruments, such as jurisprudential guides, persuasive decisions, three‑member panel decisions and reasons of interest, which consist of individual decisions that are utilized for strategic adjudicative purposes.

Collectively, these tools are intended to facilitate efficient and fair decision-making that is responsive to developments in both jurisprudence and the evidentiary foundations on which decisions are based.

The development and implementation of a new adjudicative strategy may be triggered by changing events or situations. The level of adjudicative strategy selected is dependent on the nature of the identified issue. The choice of tool will depend on the nature of the quality issues and the immediacy required to take corrective action. Adjudicative strategies can respond both to the procedural aspects of hearings and to changing immigration and refugee intake patterns and trends. In some cases, it may mean the revocation of an existing adjudicative tool to reflect higher court decisions or changes in adjudicative/operational priorities. The ongoing review and adjustment process allows for continuous monitoring and identification of solutions that may improve the quality of the IRB’s proceedings and decision-making.

Currently, adjudicative strategies are identified by the Adjudicative Strategy Committee for the RAD​, and by regional Adjudicative Consistency Groups (ACG​s) for the RPD. In cases where a strategy is needed for multiple divisions, it may be identified by the Adjudications and Operations Committee.

6.4 Individual remedial programs

As part of the Public Service Performance Management (PSPM) program for public service employees (RPD​ members), performance reviews are a continuous discussion throughout the year, during which both good performance and issues are addressed. As part of Performance Management (PM), learning and development plans, and in some cases performance improvement plans (individual remedial programs), are established.

The Strategic Human Resources Management (SHRM) Directorate includes PM, Talent Management (TM), and Learning. The PM/TM team helps with all aspects of PM and TM and, in collaboration with the learning team, supports managers in developing learning and development plans. Official languages, succession planning, and training are other components related to PM/TM.

A learning and development plan must be developed for each public service employee and includes both the employee’s career goals and aspirations, as well as planned activities that support the achievement of the employee’s established work objectives, expected behaviours and continuous development.

A performance improvement plan must be initiated when employee performance does not meet expectations, or any time the manager determines that a performance issue needs to be documented. A performance improvement plan must include:

• Specific areas for improvement;
• Actions that will be taken by the employee to address the identified areas needing improvement;
• Description of the support that will be provided by the manager to the employee to improve performance; and
• A timeline and milestones against which to measure the employee’s progress.

The performance improvement plan is a means of addressing individual quality issues. It is usually decided by the member’s immediate manager and could include a range of activities, such as specific professional development training courses, in-house training, mentorship, or returning to New Member Training. Remedial programs reactively address key issues in individual member performance to remediate member specific quality issues.

7.0 Stakeholder engagement

The IRB’s external stakeholders provide on the ground perspectives of those who appear before the Board, enhancing the development and review of our initiatives and processes. Sustained and proactive stakeholder engagement helps keep the IRB accountable, responsive and innovative. Additionally, internal stakeholders and personnel, including member cohorts, Legal Services and Tribunal Services, provide feedback on an ongoing basis to support continuous program improvements, as well as improvements to adjudicative and other support tools. As such, stakeholder input and perspectives are sought during the various stages defined in the Quality Assurance Framework. This approach helps strengthen the development, review and monitoring of policies, initiatives and service standards programs.

The Policy, Engagement and Outreach Division is responsible for external stakeholder engagement. To this end, the IRB leverages its already existing stakeholder networks in order to facilitate and maximize its consultations on the various elements of its quality programs. These networks include:

• National stakeholders: The IRB has an established consultative committee (IRBCC) which is the Board’s primary stakeholder channel through which to consult and inform on initiatives and procedures. The IRBCC meets at least twice annually as well as on an ad hoc basis to consult on specific initiatives. The IRBCC’s member organizations consist of counsel associations (lawyers and immigration consultants), Ontario’s provincial legal aid program and refugee advocate organizations.
• Regional stakeholders: Each division also maintains its established regional consultative co​mmittees with its local stakeholder groups. Its members are often associated with national stakeholder organizations and are especially helpful when engaging on operational matters where regional differences should be considered.

Additionally, in developing and monitoring its qualit​y programs, the IRB engages with portfolio partners and subject matter experts (SMEs):

• Portfolio partners: The IRB frequently consults its federal immigration and refugee portfolio organizations – Immigration, Refugees and Citizenship Canada and the Canada Border Services Agency – to provide input on operational matters. This is done with the clear understanding of the IRB’s institutional and adjudicative independence.
• SMEs: The IRB frequently consults with SMEs, which include counsel, NGOs and academics who have specialized knowledge and leading-edge experience on specific topics. With regard to the Board’s quality measures, SMEs can provide input on considerations outside the Board’s scope of expertise. SME input helps ensure that the Board’s approaches are informed, respected and cutting-edge.

Comprehensive and sustained external stakeholder engagement helps strengthen the IRB’s quality programs while also facilitating early buy-in for new initiatives.

From an internal perspective, continuous improvement relies on frequent engagement of members and staff, as well as collaborative relationships with Legal Services and Tribunal Services. This document highlights many of the mechanisms used by the IRB to ensure that engagement is ongoing and feedback is received and incorporated. Examples include the use of reasons review by Legal Services, Adjudicative Consistency Groups, Performance Management, quality centres and internal evaluations.