Printable version (PDF, 609
KB)
On this page
1.0 Introduction to the Immigration and Refugee Board of Canada's Quality Assurance Framework
1.1 Overview and objectives
The Immigration and Refugee Board of Canada (IRB) has a statutory mandate of resolving immigration and refugee cases efficiently, fairly and in accordance with the law. To ensure that the
IRB is well positioned to respond to the changing global migration patterns and unprecedented volume of people seeking access to Canada’s immigration and refugee protection system, the
IRB has established a multi‑year Growth and Transformation agenda. The agenda was announced in 2019–20 and is focused on three strategic objectives: improved and sustained productivity; enhanced quality and consistency in decision-making; and strengthened management, with a focus on people management.
As the
IRB grows, it is essential that it maintain its reputation for high‑quality and fair decisions. The
IRB's Quality Assurance Framework provides an overview of the various activities, processes, strategies and structures that contribute to quality decision-making, as it relates to adjudication, across the
IRB’s four divisions.Footnote 1 In the judicial setting, courts frequently speak of the need for administrative decision‑making to be transparent, intelligible and justified. These are the elements that mark a quality decision. Consequently, the
IRB defines quality decisions as those which can be shown to be transparent, intelligible and justified. The activities outlined in this Framework are therefore closely aligned to these elements and create a path for the
IRB to follow towards its strategic objective of enhanced quality and consistency in decision-making. To this end, several principles guide the
IRB and decision-makers as they pursue this objective.
1.2 Guiding principles
The following principles guide quality assurance at the
IRB:
Compliance: The necessary adherence to legislation, regulations and policies to fulfill the
IRB’s mandate.
Consistency: The belief that the decision-making process and the factors that are considered should be applied consistently.
Equity: The belief that all cases should be treated in a fair and impartial manner, in accordance with the law.
Independence: Quality assurance strategies support the accountability of members and facilitate the independence of member’s decision making.
Collaboration: Quality assurance is a shared responsibility across
IRB management and members.
Ongoing process of improvement: Quality assurance is not an end result but an integral component of the decision-making processes, with a commitment to the continual improvement of quality assurance strategies and tools.
1.3 Approach
The
IRB Quality Assurance Framework is organized according to the stages of a continuous improvement cycle: Plan, Do, Monitor and Measure, and Adjust (see fig. 1). Although many of the activities across the stages run concurrently, there is a natural flow of progression.
Plan: This stage ensures that activities and investments in support of quality decisions are reviewed and prioritized at the onset of the year and revisited at least semi-annually in order to ensure continued alignment with corporate quality and performance objectives. Quality planning focuses on training, mentoring, performance targets and evaluations, which are all aimed at ensuring that best practices are in place for continued improvement. Additionally, updates to adjudicative strategies occur periodically as part of formal legislative reviews and support improved consistency in decision-making.
Do: This stage addresses all the activities which directly support quality decisions in an operational context, where operations at the
IRB are comprised of decision‑makers across four divisions: the Refugee Protection Division (RPD), the Refugee Appeal Division (RAD), the Immigration Division (ID) and the Immigration Appeal Division (IAD). Activities include recruiting and training new members, as well as providing ongoing training and mentorship, to ensure continued professional development (PD) for all members. Further focus is put on ensuring that decision‑makers have collaborative forums to engage with their colleagues on current issues and challenges, and that they have the collaborative tools to facilitate the sharing of information.
Monitor and Measure: This stage is about monitoring and measuring the effectiveness of what was defined in the “Plan” stage and performed in the “Do” stage. It answers questions about the effectiveness of our training and mentorship programs. It reviews the quality of decisions, ensuring we have the right practices in place to support decision-making excellence, including timely and complete pre-proceeding readiness; respectfulness of proceedings; focused proceedings; and clear, complete, concise and timely decisions. The activities at this stage are facilitated both at the division level, where quality centres are being established, and centrally, where independent third‑party reviews are facilitated.
Adjust: The final stage of the Framework is comprised of activities to address the changes that need to be either incorporated immediately into operations (the “Do”) or included as input into the next planning cycle. This includes activities such as the development of action plans to address findings from third‑party reviews and external audits, revisions to training programs, and adjustments to adjudicative strategies in response to changing trends. Finally, this stage provides for defining remedial actions to support individual members in their professional development.
Fig. 1:
IRB Quality Assurance Framework for Quality Decision-Making
*New initiative to be implemented in FY2020-21
Text format: Fig. 1:
IRB Quality Assurance Framework for Quality Decision-Making
The
IRB’s Quality Assurance Framework for decision-makers is depicted by a series of five chevrons sequentially aligned to illustrate how each section of the Framework leads to the next as a cycle of continual improvement. The sections are
Plan,
Do,
Monitor & Measure, and
Adjust and contain the key activities of each section. Finally, all sections are underpinned by Stakeholder Engagement which takes place throughout the process with specific impacts to the activities under the
Plan and
Do sections.
The
Plan section includes the following activities:
- Identify and prioritize training and mentoring needs
- Identify and prioritize quality evaluation and audit plan
- Identify adjudicative strategies
The
Do section includes the following activities:
- Merit-based recruitment
- Training for new members
- Professional development
- Mentorship
- Reasons review by Legal Services
- Adjudicative and support tools
- Adjudicative Consistency Groups
- Specialized teams/training, specifically Gender-related claims. (Note: new initiative to be implemented in fiscal year 2020-21)
The
Monitor section includes the following activities:
- Performance management
- Internal and external reporting
- Appeal and/or judicial review
- Review higher court decisions
- Quality Centres (Note: new initiative to be implemented in fiscal year 2020-21)
The
Measure section includes the following activities:
- Third-party reviews
- Internal evaluations
- External audits
- Results of member complaints
The
Adjust section includes the following activities:
- Develop and implement management action plans (Audits, third-party reviews)
- Revise training programs
- Develop and implement adjudicative strategies
- Individual remedial programs
1.4 Roles and responsibilities
The
IRB’s leadership plays an important role by promoting and emphasizing innovation and teamwork, which provides an important foundation for implementing the Quality Assurance Framework. However, lasting results cannot be achieved without the commitment and support of the entire organization. Within this collaboration, several groups play integral roles in upholding quality assurance:
Chairperson: The Chairperson provides overall leadership and direction with respect to quality assurance and professional development.
Deputy chairpersons: The deputy chairpersons are accountable to the Chairperson for the quality decision-making of the members assigned to their divisions. They provide vision and set priorities with regard to quality programs, and enable assistant deputy chairpersons to implement these priorities.
Assistant deputy chairpersons: Assistant deputy chairpersons provide leadership in implementing quality programs and ensuring that appropriate resources are committed to support quality decision-making. They create an environment for their teams that facilitates and fosters continuous improvement and quality decision-making. Representative assistant deputy chairpersons from each division will play an integral role in the management of the quality centres.
Quality centres: The quality centres will provide focused attention on initiatives, tools and mechanisms that improve quality. They will also provide a mechanism to ensure that best practices are communicated and that quality issues are identified and addressed in a timely manner, independent of the individual performance management system.
Coordinating members: Coordinating members support and encourage members’ personal development by providing regular feedback and identifying areas for development with respect to quality decision-making. They provide advice on procedural and other issues, identify training needs or areas of inconsistency in member performance, and promote the use of best practices.
Members: Members are responsible for appropriately applying legislation, regulations and policies, and for employing strategies and tools to render quality decisions. They also participate in learning and personal development opportunities for the purpose of enhancing competencies, thereby strengthening quality decision-making.
Legal Services: Legal Services prepares jurisprudential updates, case notes and other essential documentation contributing to quality decision-making. Legal Services is available to assist members by addressing legal questions and preparing quality reasons. It is also involved in the training of members.
Strategic Directions and Corporate Affairs Branch: The Strategic Directions and Corporate Affairs Branch maintains the Quality Assurance Framework. It is responsible for many of the monitoring and measuring activities, and the reporting of results.
2.0 Plan
2.1 Identify and prioritize training and mentoring needs
The identification of training and mentoring needs is an ongoing process for the
RPD and the
RAD. Training and mentoring needs are identified so that members receive training and mentoring support where it will be most impactful to ensure quality adjudicative decision-making. Within the
RPD and the
RAD, coordinating members (supervisors) consider member requested needs, data trends, performance information and stakeholder feedback when designing new member training, developing training materials, establishing mentorships and holding professional development (PD) sessions.
Regionally based
PD committees establish forward agendas for
PD training based on current issues and needs identified in their regions. For example, for the
RPD, each
PD committee is composed of members and legal advisors, and is chaired by a coordinating member. The committee meets bi-weekly or monthly to identify training needs and to plan
PD sessions. The committee is responsive to training needs identified by
RPD management and Legal Services for members, and provides the forum for such training in its regularly scheduled
PD sessions.
At the
RAD, a national
PD training committee sets learning objectives for the year to ensure ongoing
PD. Priorities are identified with input from the Adjudicative Strategy and the
PD committees, and emerging Federal Court jurisprudence. Other inputs are received from Legal Services and management action plans (MAPs) developed in response to quality measurement initiatives (QMIs)—the latter is facilitated by the Strategic Planning, Accountability and Reporting (SPAR) Directorate.
Mentors and needs for mentorship are identified within each division of the
IRB and take three forms: 1) New member mentorship (peer-to-peer colleague support [“buddies”] and assigned mentors); 2) Ongoing
PD (mentorship requested by a member to improve a specific skill); and 3) Remedial/reactive mentorship (based on an identified performance need). With the exception of peer‑to‑peer mentorship, mentors are generally hired on contract from amongst former
IRB members.
The Member Learning Secretariat (MLS) plays a key role in supporting this work. The goal of the
MLS is to provide national, organization-wide coordination of learning and
PD activities for decision-makers such as new member training,
PD sessions, mentoring and national training symposiums. This support comes in the form of planning training and learning activities, preparing training material, coordinating with regional counterparts for venues and equipment, tracking training sessions, etc. The
MLS is also building capacity to be able to support the identification of training needs by employing qualitative assessment tools such as surveys.
2.2 Identify and prioritize quality evaluation and audit plans
The planning of the
IRB’s third-party reviews and internally led evaluations helps to ensure that all components of quality decision-making are reviewed, and that areas for continuous and ongoing improvement are identified.
The evaluation unit within the Strategic Planning, Accountability and Reporting (SPAR) Directorate conducts an annual planning exercise to identify and prioritize all third-party reviews, as well as internally and externally led evaluations over a five-year period. This process is conducted in accordance with the Treasury Board of Canada Secretariat (TBS)
Policy on Results and is facilitated through the use of the
TBS’s
Evaluation Coverage Tables.
As part of this exercise, some reviews such as quality measurement initiatives (addressed in
Section 5.1 - Third-party reviews) are planned for each division on an annual basis, while other reviews are planned at the request of internal
IRB stakeholders, the
TBS or Parliament. Reviews that are initiated internally seek to ensure the quality of initiatives that support adjudicative decision-making, as well as the quality of the decisions themselves.
The proposed list of prioritized evaluations and audits are reviewed and approved by the Audit, Evaluation and Performance Measurement Committee (AEPMC).
The
IRB rarely initiates or conducts its own audits; rather, the audits are conducted by the Office of the Auditor General (OAG) or the Office of the Comptroller General (OCG). The coordination of responses to these audits and the development of Management Action Plans (MAPs) is the responsibility of the
SPAR Directorate and is integrated into the
SPAR Directorate’s plans.
2.3 Identify adjudicative strategies
At a foundational level, adjudicative strategies guide the
IRB as a tribunal and drive consistency and quality in decision-making by ensuring decision‑makers (members) keep the
IRB’s “raison d’être” front of mind. While each decision‑maker is independent, adjudicative strategies are used to model best practices, to provide examples of sound decision‑making strategies, to explain complex areas of law and to explore unsettled questions of fact.
The Board’s adjudicative strategy initiatives can be loosely grouped into two categories:
-
soft law instruments, for example guidelines, policies and practice notices; and
-
adjudicative instruments, such as jurisprudential guides, persuasive decisions, three‑member panel decisions and reasons of interest, which consist of individual decisions that are utilized for strategic adjudicative purposes.
Collectively, these tools are intended to facilitate efficient and fair decision-making that is responsive to developments in both jurisprudence and the evidentiary foundations on which decisions are based.
Adjudicative strategies are identified and implemented in order to enhance quality, consistency and efficiency in decision-making. For example, where the
RAD uses one of its adjudicative tools to map out a legally sound framework for analysis that can be used in a large number of cases, this framework can then be used by other decision-makers to render legally sound decisions in an efficient manner. The framework does not dictate a result, or infringe independence, but it does foster a consistent approach to adjudicating similar cases. Decisions with respect to the use of adjudicative tools are driven by a number of factors, including assessments of common case types, unresolved legal and factual issues, and country volumes. The use of these tools is developed in collaboration with adjudicative strategy committees and their respective divisions. Once designated, these tools are evaluated and monitored based on their ongoing utility to members and their continued accuracy and relevance. Examples of existing adjudicative strategies include jurisprudential guides, persuasive decisions and reasons of interest, which are published online.
Planning related to adjudicative strategies occurs on an ongoing basis through the use of metrics to support the forecasting of legal issues or to identify areas of inconsistency, for example. This allows the
IRB to map out particular topics and determine if use of an adjudicative tool or strategy may be warranted. Additionally, key legislative changes (including to the Rules) trigger significant planning events to review, improve or adjust our adjudicative strategies. More flexible policy instruments, such as practice notices, are used to make minor adjustments, if required, between major legislative reviews. (More information on this aspect is provided in
Section 6.3 – Develop and implement adjudicative strategies.)
3.0 Do
3.1 Merit-based recruitment
The
IRB has developed staffing strategies and tools to ensure future decision makers are able to render fair and efficient decisions. During the hiring process, applicants must successfully complete a written exam and an interview that are tailored to measure decision-making ability, alongside other necessary qualities, such as judgment and reasoning, information-seeking and self-control. A reference check and background check are also administered to verify the candidate’s personal suitability for the member position, and a values and ethics assessment is conducted. The staffing strategy is responsive to data trends, such as determining the predicted number of applications needed to fill a set number of positions.
For the
RPD, merit-based recruitment is in place to ensure that persons hired are qualified and able to render quality decisions. At the
RAD, in collaboration with Governor in Council Secretariat Services, a list of qualified persons is presented to the Minister for appointment by the Governor in Council (GIC).
3.2 Training for new members
Led by experienced member trainers and co-delivered with Legal Services counsel, New Member Training (NMT) combines enhanced pedagogy with teaching to the theory of the case, helping members focus on determinative issues and deliver quality decisions. It is built on the principles of adult education training, and incorporates hands-on member experience through job shadowing and participating on three-member panels. This training runs for approximately seven weeks, and ensures that all new members have a standardized approach to conducting hearings and rendering decisions.
NMT is regularly updated to reflect changes in policy, procedures and law. Lessons learned are created at the end of each session to promote the quality and integrity of the training. The training approach was developed with help from a third-party consultant (a professor from Saint Xavier University with expertise in adult learning).
A Member Learning Secretariat (MLS) was created in spring 2019, as a resource for all divisions to support certain facets of the administration of member training. The goal of the
MLS is to provide national, organization-wide coordination of learning and professional development activities for decision-makers, including
NMT. This support comes in the form of planning training and learning activities, preparing training material, coordinating with regional counterparts for venues and equipment, tracking the training sessions, etc.
NMT ensures that all new members receive the knowledge, tools and support required to develop their decision-making skills to a level where their decisions are of high quality.
3.3 Professional development
Monthly Professional Development (PD) sessions provide an opportunity for members to reflect on pressing substantive and legal issues, including feedback resulting from
RAD decisions and/or judicial reviews of the Federal Court. A typical
PD session includes a jurisprudential update by Legal Services on recent case law and a member-led component on a selected topic. These monthly sessions provide continuous learning opportunities for all members on matters most pertinent to their work so that they are well informed about current issues and better equipped to make quality decisions.
For the
RPD, currently each region develops its own
PD sessions based on emerging issues. In each region,
PD sessions are organized by a committee that consists of up to five members, a coordinating member and a Legal Services advisor. The
PD committee solicits interest from members about topics on which they would like to have sessions and also generates its own ideas for
PD sessions. Additionally, the
PD committee is responsive to training needs identified by
RPD management and Legal Services for members. In some regions, a reasons writing committee exists that is focused on providing tools and training for members to render concise and timely decisions.
PD at the
RAD is guided by a national
PD committee, in collaboration with the Deputy Chairperson’s Office.
PD sessions are offered nationally in both languages. Additionally, informal regional
PD sessions are frequently conducted by regional assistant deputy chairpersons.
PD sessions cover a wide range of topics from jurisprudential and policy updates, to issues of hearing room management, to the finalization of less complex claims, and the efficient rendering of decisions. Additionally, they cover topics such as avoiding burn-out, sensitivity fatigue, resilience and other areas that address the mental health and wellbeing of members. These topics contribute to better working environments, which result in a better quality of work.
3.4 Mentorship
Mentorship aims to ensure that members have access to support structures. In the case of remedial mentorship, it also allows for the targeting of identified performance challenges of specific members. Mentoring support reinforces training material and helps members develop and maintain the skills needed to make high quality decisions.
Mentorship at the
IRB takes three forms: 1) New member mentorship (both peer-to-peer colleague support [“buddies”] and assigned mentors); 2) Ongoing professional development (mentorship requested by a member to improve a specific skill); and 3) Remedial/reactive mentorship (based on an identified performance need). With the exception of peer to peer mentorship, mentors are generally hired on contract from amongst former
IRB members.
The
RPD New Member Buddy Initiative supports new members by informally providing opportunities to discuss case preparation and the decision-making process with experienced members. The initiative allows new members to be exposed to various ways of organizing their workload and practical matters such as managing administrative tasks and procedural issues in the hearing room.
The
RPD’s proactive National Mentorship Program helps new members transition into a full hearing schedule and implement lessons learned from training. This provides support for new members during the first four to six weeks of decision-making. It is the next step in adjudicative development after New Member Training and three member panels. The program’s goal is to create and maintain the skills needed to support fair, efficient and high-quality adjudication.
As part of the
RPD Member Mentorship Program,
RPD mentors address identified areas for improvement in a member’s performance. Mentors closely review files with members, observe hearings and read finalized decisions.
RPD mentors can be requested by either the member, or assigned by management after performance evaluations.
RPD mentors contribute to the success of new members, as well as the growth of more experienced members, by enhancing the quality of their work through coaching.
The
RAD has implemented internal (current members) and external (former members) mentoring programs. Mentorship in the
RAD is similar to that in the
RPD; it is provided proactively to all new members. It is continued for varying amounts of time depending on the new members’ needs, and it is offered remedially in situations where particular performance needs have been identified.
3.5 Reasons review by Legal Services
Legal Services counsel is available to assist members in preparing quality reasons that accurately reflect the rationale for their decisions and offering advice that adopts a legal risk approach to possible
RAD or Federal Court review. Under the
IRB’s
Reasons Review Policy and
Instructions Governing Solicitor-Client Privilege, reasons reviews by counsel are confidential to members, and voluntary, although an exception is made during the training period so that management can be apprised of any training needs. Members are assigned a Legal Services advisor who is available to address any legal question, whether it be case specific or a broader adjudicative/legal question.
While reasons review is generally an optional service offered to decision makers, the training of new members is a special situation, in which legal counsel work with them as part of a team. This team may be composed of a professional development advisor, an experienced member, acting as mentor, and the new member's manager. Therefore, new members are expected to make use of the reasons review service offered by Legal Services in accordance with the New Member Training Program.
Additionally, a member manager (coordinating member) may recommend to a member that they should have Legal Services review their draft reasons because of the complexity or novelty of the case or because the member manager is of the view it would be consistent with the objectives of reasons review.
When providing reasons review, counsel seek to promote quality, efficiency and independence along with consistency in their advice, including in providing timely advice to members seeking to render oral decisions. However, how information is leveraged is at the discretion of the member, as the decision and reasons remain their own. The decision maker is free to accept or to reject comments made by Legal Services on the reasons. Legal Services does not share any specific cases or concerns with management, but may raise frequently noted issues and trends, for purposes of identifying general training needs.
3.6 Adjudicative and support tools
Adjudicative and support tools are developed to facilitate quality decisions by improving procedural regulations, information access, and/or horizontal member engagement. As described in
Section 2.3 – Identify adjudicative strategies, these tools, which support adjudicative strategies, lead to improved quality through enhanced consistency in decision-making.
Adjudication tools are primarily written guidance for members—and in some cases the Registry, or others involved in proceedings before the
IRB—dedicated to improving the quality of
IRB regulations that assist members in achieving their mandate.
Adjudicative tools include:
- The Chairperson’s guidelines, which provide guiding principles for adjudicating and managing cases;
- Jurisprudential guides, which support consistency in adjudicating cases which share essential similarities;
- Policies, which are formal statements that explain the purpose and the mechanics of operational initiatives at the Board; and
- The Chairperson’s instructions, which provide formal direction that obliges specific
IRB personnel to take or to avoid specific actions.
Adjudicative tools can also cover specific situations (e.g.
Guideline 9: Proceedings before the
IRB Involving Sexual Orientation and Gender Identity and Expression or
jurisprudential guides).
Members make use of adjudicative tools to reach their own conclusions based on the facts of each particular case. Although not binding, members are expected to follow guidelines, unless compelling or exceptional reasons exist to depart from them.
Support tools are primarily concerned with improving information access and horizontal member engagement, which subsequently enhances decision quality. The Knowledge Management Tool (RPD only), up-to-date country statistics and
national documentation packages, and
responses to information requests are all tools that support information accessibility and also help managers with monitoring trends. The
RAD’s Members-Only Collaborative Workspace is a support tool that improves horizontal engagement of members by enabling online consultation and feedback. The
RAD member tool provides an electronic forum for members to exchange information and ask questions about issues of mutual concern.
These tools aim to ensure both the consistency and the quality of decision-making and proceedings more broadly at the Board and allow members to efficiently and consistently consult knowledge that assists them in carrying out their duties.
3.7 Adjudicative consistency groups
RPD members meet in adjudicative consistency groups (ACGs) to discuss adjudicative trends and issues related to emerging country conditions, claim type analysis, case law, etc., in an effort to improve adjudicative quality and consistency in divisional decision-making. This allows a forum for members to discuss and understand trends and issues in depth and in a consistent manner.
ACG meetings are Consolidated-Bathurst type discussions led by a member. These meetings typically last about two hours and occur once a month. They are designed to enhance the consistency and quality of the decision-making objective of the
RPD.
ACG meetings allow all members adjudicating the same countries to have plenary discussions on issues of prevailing country conditions, policies, procedures and law, while respecting the adjudicative independence of the participating members. Members discuss these issues amongst themselves, share their experiences and ask questions of their colleagues. Highlights from these conversations are captured in summary reports (Feedback Forms) that provide a record of the major issues discussed, the material that was provided, and any follow up that may be required.
The objective of
ACG meetings is to ensure that members are fully aware of all the issues, trends and developments in the countries from which they hear claims.
ACG meetings are not meant to be prescriptive about what decisions members should make in their hearings.
ACG meetings help to address matters of intra-team variance related to both decision-making and the efficient use of hearing room time.
3.8 Specialized teams/training
Specialized teams and task forces have been used for various reasons in the past (to address backlogs or specific regional issues, for example) and are currently being created for quality purposes.
Specialized teams (whether divisional or regional), and the related specialized training, allow the
RPD to develop advanced subject matter expertise in areas related to country conditions, claim types, questioning techniques, and dealing with sensitive claims and vulnerable claimants. Specialized teams afford the
RPD opportunities to test quality improvement and adjudicative strategy initiatives and to receive timely feedback. New emerging issues can also be identified efficiently.
Country specialization is a foundational principle for the assignment of cases to
RPD members in the Central Region. Teams tend to be specialized in a set number of countries, and within each team there can be further specialization, which allows members to develop an in-depth understanding of country conditions and claim types. Members are therefore well placed to recognize patterns and respond appropriately when country conditions quickly change. Country specialization also leads to greater efficiency within the hearing room and in the rendering of decisions.
Country specialization considers organizational needs but also members’ expertise and preferences. Where a limited volume of cases precludes country specialization, countries are assigned based on regional or thematic specialization.
4.0 Monitor
4.1 Performance management
When it comes to managing the performance of employees and members, the
RPD follows the Public Service Performance Management (PSPM) Cycle, as set by the Treasury Board of Canada Secretariat (TBS). This includes setting performance objectives at the beginning of the fiscal year, and conducting mid year and end of year reviews. It also includes an assessment of the member against the four core competencies of the Public Service. In addition, members receive regular reports on their performance in terms of timeliness of decisions and productivity, and meet with their managers on a monthly basis to discuss their performance. The member Code of Conduct is included as a measure in these reviews. In addition, coordinating members (supervisors) will regularly review decisions as part of this cycle and will, on occasion, pull audio files to listen to how hearings are conducted. Any identified issues are addressed on an ad hoc basis, as well as in a more structured fashion during the formal performance cycle.
Supervisors build quality into performance management agreements at the beginning of each year. While there is not uniform language regarding quality, there is a nationally agreed upon approach for performance expectations (120/year, 90% finalized within 25 days, 100% finalized within 50 days).
For new members in the
RPD, there is an onboarding process that includes staged assessments. This process usually takes six to eight months. A template is filled in for each new member for each of two observations at each assessment stage, and four decisions are read for each new member at each assessment stage. The completed templates are then consolidated into a single report per assessment stage for each member and then discussed in a committee consisting of all those who are involved in training and onboarding. These meetings permit additional training, coaching or mentoring needs and measures to be identified and implemented. The process is overall robust and has produced good results, and because the program is so comprehensive, should the need arise to end a member’s contract within the probationary period, the
RPD would be in a good position to do so.
Given that
RAD members are
GIC appointees, they do not fall under the
PSPM; however, the
RAD follows a separate, similar formal review cycle, with annual and end of mandate reviews.
The Strategic Human Resources Management (SHRM) Directorate provides support on a variety of aspects of the organization, including Performance Management (PM) and Talent Management (TM). The
PM/TM team provides support to the entire
IRB, including the
RAD and the
RPD. It also acts as the link between the
IRB and the
TBS, which requires reporting on
PM and
TM through the Management Accountability Framework.
The numerous means available to assess, motivate and manage
IRB members ensure that decisions, reasons and hearing room conduct are of high quality.
4.2 Internal and external reporting
Regular monitoring of the
IRB’s progress against priorities, targets and other key activities ensures the
IRB is able to recognise challenges and continually make adjustments in order to meet our quality objectives. Regular reporting and publishing of these reports also ensures a level of transparency and accountability with respect to the
IRB’s activities, including results, audits, etc.
Internal reporting related to quality is coordinated by the Strategic Planning, Accountability and Reporting (SPAR) Directorate through regular updates to the Audit, Evaluation and Performance Measurement Committee and the Executive Management Board. These executive level committees review results of audits and evaluations, as well as updates on priority initiatives, including quality. Additionally, the Standards, Analysis and Monitoring (SAM) unit produces regular reports that provide information on divisional performance, including quality.
External reporting related to quality is also coordinated by the
SPAR Directorate through public reports to Parliament such as the Departmental Plan (DP) and the Departmental Results Report (DRR), which are submitted to Parliament and published on the
IRB’s external website. These public reports indicate plans and results related to the program targets of the
IRB, one of which is quality of decision-making. The reports indicate the planned and actual percentage of cases that meet quality standards.
The results that are indicated in the
DRR are based on the
IRB’s quality measurement initiative (QMI) whereby a third-party reviewer assesses a sample of decisions from each of the four divisions (RPD,
RAD,
ID and
IAD) against a checklist of procedures that ensure quality decision-making. More information about
QMIs is available in
Section 5.1 – Third-party reviews.
Regular reporting helps to determine the
IRB’s overall results for decision-making excellence, including: timely and complete pre-proceeding readiness; respectful proceedings; focused proceedings; and clear, complete, concise and timely decisions.
4.3 Appeal and/or judicial review
An appeal to the
RAD or a judicial review, with leave, at the Federal Court are legal remedies where eligible parties to the initial decision think an error has been made and/or believe that a breach of natural justice has occurred. These function as corrective measures and may also be seen as qualified indicators of quality decision-making when decisions at either division are upheld.
The
RAD reviews decisions of the
RPD when the claimant or the Minister believes that an error was committed by the
RPD or that a breach of natural justice occurred, or wants the
RAD to review the
RPD decision in light of information that could not have been presented to the
RPD prior to the
RPD decision.
The
RAD decides to either:
- confirm the determination of the
RPD;
- set aside the determination and substitute a determination that, in its opinion, should have been made; or
- refer the matter to the
RPD for re-determination, giving the directions to the
RPD that it considers appropriate.
When the Federal Court decides to judicially review the reasons for the refugee appeal decision, or in limited circumstances, the refugee protection claim decision, it will decide whether the decision was reasonable. A hearing where both the claimant or appellant and the Minister’s counsel provide arguments is part of this process. The Federal Court may grant or dismiss the judicial review. If it is granted, the matter is returned to the
IRB for re-determination.
The
IRB may decide to seek leave to intervene in a higher court proceeding to explain to the court the Board’s adjudicative strategy, to make submissions on jurisdiction or to explain operational considerations. In any case in which the
IRB seeks leave to intervene, the impartiality of the Board must be maintained.
Summaries of
RAD and Federal Court decisions are regularly shared with members by Legal Services. These summaries allow members to be updated on how the appellate bodies view the findings of the
RPD. Any decisions of the
RAD and the Federal Court that are of particular significance to the
RPD are discussed in greater detail at Professional Development sessions.
4.4 Review higher court decisions
Federal Court decisions are reviewed by
RPD and
RAD management and Legal Services to assess whether they identify particular Professional Development needs or member-specific concerns.
Legal Services provides jurisprudential updates on Federal Court and
RAD decisions as well as develops specific training in light of new or novel jurisprudence.
A Higher Court Decisions database is maintained with case notes of decisions dealing with refugee and immigration matters. Most of the case notes are from decisions rendered by the Federal Courts of Canada. However, it also contains selected decisions of other courts (e.g. the Supreme Court of Canada and provincial courts). The application includes case notes of decisions rendered since July 22, 1996. The case notes are prepared by the
IRB’s Legal Services. Each case note includes a string of keywords and a hypertext link that leads to the full-text reasons, when available.
4.5 Quality centres
The quality centres in the
RPD and the
RAD are being established in fiscal year 2020–21 to monitor the quality of decision-making. When fully implemented, quality centre data analysts will mine
RAD and
RPD decisions, as well as Federal Court cases in order to identify specific legal questions that repeatedly occur. The Adjudications Operations Committee will meet regularly to discuss quality centres and adjudicative strategies to address these issues. Joint
RPD/RAD meetings will also be held to discuss legal issues and approaches to questions concerning refugee determination.
The quality centres will provide focused attention on initiatives, tools and mechanisms that improve quality, such as improved training, mentorship, performance management and adjudicative strategies. They will ensure that legal issues in decisions are identified early on, and appropriate adjudicative strategies are adopted in response. They will provide a mechanism to ensure that best practices are communicated and that quality issues are identified and addressed in a timely manner, independent of the individual performance management system. For example, if a quality centre noticed a trend, such as an increase in the postponement rate due to council not being able to access certain documentation, it would trigger a discussion on how the issue could be addressed, with the goal of encouraging consistency and adherence to rules. In this case, a new adjudicative strategy might be proposed, such as the issuance of a practice notice.
5.0 Measure
5.1 Third-party reviews
Third-party reviews refer to quality measurement initiatives (QMIs) and external evaluations. In these cases, a third-party assessor is recruited to maintain objectivity and conduct reviews on the quality of initiatives and decision-making. Third-party reviews of
IRB programs, processes and procedures help to identify strengths and areas for improvement by measuring the quality of initiatives that support decision-making, and also help to measure the quality of decision-making itself.
The Strategic Planning, Accountability and Reporting (SPAR) Directorate leads annual
QMIs, which assess the quality of decision-making within each
IRB division, through the use of a third-party assessor who follows a standardized approach. The assessments are conducted on a sample of cases for a given division and are measured against a set of standards for quality decision-making. Within each
QMI, Gender-Based Analysis Plus (GBA+) is also conducted, as per guidelines by the Treasury Board of Canada Secretariat which indicate that GBA+ must be conducted for all new initiatives and evaluations. Strengths and areas for improvement are identified in the final report and recommendations are provided. The divisions use the findings to support improvements to their programs and, more specifically, to the quality of member procedures and decisions making. External evaluations and other reviews are often initiated as a result of findings from other processes (complaints,
QMIs or other external influences).
A report with recommendations is produced following all third-party reviews. The
IRB responds with a Management Action Plan (MAP), which is monitored by the
SPAR Directorate and updated by the office of primary interest. Reports and
MAPs are presented and monitored through the Audit, Evaluation and Performance Measurement Committee (AEPMC), which is chaired by the
IRB Chairperson. Examples of
MAPs related to quality assurance include: improved support for the planning, design and delivery of member training programs; and provision of professional development refreshers regarding member questioning.
Third-party reviews are a necessary component of assuring quality within the
IRB. While audits function to assess the priorities of external stakeholders, third-party reviews function to assess the priorities of internal stakeholders. Third-party reviews are coordinated internally, but the evaluative component is outsourced to ensure that final reports remain objective.
5.2 Internal evaluations
An evaluation in the Government of Canada is a systematic and neutral collection and analysis of evidence to judge merit, worth or value. It determines the extent to which a program or project has achieved expected results. Evaluation informs decision-making, improvements, innovation and accountability.
The Evaluation Unit within the Strategic Planning, Accountability and Reporting (SPAR) Directorate supports the
IRB divisions or branches in assessing the quality of new and ongoing initiatives/programs. The
SPAR Directorate facilitates the development of logic models and theory of change models against which success can be measured. In a leading role, and in consultation with the Standards, Analysis and Monitoring (SAM) unit, the
SPAR Directorate supports the development of performance measures and methodologies for collecting data to inform performance. The
SPAR Directorate analyzes the results, and reports findings to the client.
Conducting evaluations improves
IRB decision quality by assessing the outcomes of new initiatives that support decision-making. For example, the
SPAR Directorate is currently working on two evaluations for a new position within the
RPD and the
RAD respectively. Each of these divisions has created a new position which seeks to support members by ensuring that case files are hearing ready, and that members have accurate and reliable information about every case, each with the end goal of fair and timely adjudication of cases. The evaluations will help determine whether these positions are successful in meeting this objective and will help inform investment decisions about these new positions going forward.
5.3 External audits
Auditing is a professional, independent and objective assurance and consulting activity designed to add value and improve an organization's operations. It helps an organization accomplish its objectives by bringing a systematic, disciplined approach to assessing and improving the effectiveness of risk management, control and governance processes.
External audits of the
IRB are typically conducted by the Office of the Auditor General (OAG) and the Office of the Comptroller General (OCG) which send out routine and non routine call letters to inform departments of new audits and the associated areas that will be assessed. The role of the
IRB, with the Strategic Planning, Accountability and Reporting (SPAR) Directorate as the lead, is to analyze the audit request, coordinate responses with offices of primary interest, and compile responses and evidence for submission to the office that is conducting the audit.
Following the
OAG/OCG’s examination, a draft report is shared with the
IRB to confirm the accuracy of their findings, review recommendations, and when necessary, respond to recommendations with Management Action Plans (MAPs). Reports and summaries are published on the
IRB website. The
OAG/OCG monitor these
MAPs and request annual or semi-annual updates. Once artefacts are provided that demonstrate that the
IRB has fulfilled the audit recommendations,
MAP items can be closed.
Independent, objective and systematic audits of the
IRB represent a vital resource for maintaining and improving decision quality at the
IRB. It presents an opportunity for the
IRB to identify program gaps and initiate departmental improvements.
5.4 Results of member complaints
Anyone may file a complaint about the conduct of an
IRB member. The complaint procedure is meant to address a members’ professional conduct. It provides a formal mechanism for counsel, any party to
IRB proceedings,
IRB personnel, an organization or the public to raise any challenges in member behaviour that they see as counter to the
IRB Member Code of Conduct. Complaints must be made in writing to the Director of the Office of Integrity. A complaint form and procedures guide are available on the
IRB public website.
Complaints against members that fall under the scope of the Member Code of Conduct are examined by the Office of Integrity, or in exceptional circumstances, an external investigator. A report of the investigation is provided to the Chairperson, who will examine it and decide whether to accept the conclusions, and communicate the outcome to the complainant, the member concerned, and the Assistant Deputy Chairperson and Deputy Chairperson. The Chairperson will decide whether any follow up actions are required. Overall system trends are monitored and used to inform updates to training programs as required.
Aggregate information regarding numbers and disposition of complaints against members is published on the
IRB’s website. Publishing aggregate information regarding results of member complaints on the website creates transparency.
6.0 Adjust
6.1 Develop and implement management action plans (audits, third-party reviews)
Management Action Plans (MAPs) or Management Response and Action Plans (MRAPs) are developed in response to recommendations indicated in audits, and third-party reviews such as quality measurement initiatives (QMIs) and external evaluations. The Strategic Planning, Accountability and Reporting (SPAR) Directorate coordinates the development of a
MAP/MRAP by engaging the office of primary interest (typically a division) to review the recommendations of an audit or third-party review. The office of primary interest will then respond by accepting or rejecting it, and providing a plan for how they will address the recommendation, including timelines.
MAPs and
MRAPs are presented at the Audit, Evaluation and Performance Measurement Committee (AEPMC) for approval. Progress against the
MAP is monitored and tracked by the
SPAR Directorate, and brought back to
AEPMC to ensure ongoing accountability of implementation.
MAPs are integral to quality improvement as they consider identified program or process gaps and specify next steps for resolving them, leading to improvements in processes, training and results. Beyond outlining a plan of action, the development of a formal
MAP ensures ongoing accountability to both internal and external stakeholders.
MAPs remain active until sufficient evidence is provided to ensure gaps have been addressed and functional quality is improved.
6.2 Revise training programs
Member training programs for new and experienced members are reviewed on a continuous basis by the divisions, in consultation with and with the support of Legal Services. Training revisions ensure that training methodology and content are up to date with best practices, and allow new tools, strategies and research to be integrated into the training program. Additionally, revisions consider whether any changes in law, findings through
QMIs, or systemic issues identified through performance management and the member complaint process warrant updates to the training program.
For the
RAD, this is an “evergreen” activity performed in the period preceding scheduled new member training (NMT). At a minimum, relevant jurisprudence that has emerged since the last delivery of
NMT is updated. Other revisions will be made subsequent to discussions of the Adjudicative Strategy and Professional Development committees, or upon the recommendations of the lead of the Quality Centre.
6.3 Develop and implement adjudicative strategies
As identified in
Section 2.0 – Plan, adjudicative strategies are central to the
IRB’s efforts to enhance both consistency and quality in decision-making. The Board’s adjudicative strategy initiatives can be loosely grouped into two categories:
-
soft law instruments, for example guidelines, policies and practice notices; and
-
adjudicative instruments, such as jurisprudential guides, persuasive decisions, three‑member panel decisions and reasons of interest, which consist of individual decisions that are utilized for strategic adjudicative purposes.
Collectively, these tools are intended to facilitate efficient and fair decision-making that is responsive to developments in both jurisprudence and the evidentiary foundations on which decisions are based.
The development and implementation of a new adjudicative strategy may be triggered by changing events or situations. The level of adjudicative strategy selected is dependent on the nature of the identified issue. The choice of tool will depend on the nature of the quality issues and the immediacy required to take corrective action. Adjudicative strategies can respond both to the procedural aspects of hearings and to changing immigration and refugee intake patterns and trends. In some cases, it may mean the revocation of an existing adjudicative tool to reflect higher court decisions or changes in adjudicative/operational priorities. The ongoing review and adjustment process allows for continuous monitoring and identification of solutions that may improve the quality of the
IRB’s proceedings and decision-making.
Currently, adjudicative strategies are identified by the Adjudicative Strategy Committee for the
RAD, and by regional Adjudicative Consistency Groups (ACGs) for the
RPD. In cases where a strategy is needed for multiple divisions, it may be identified by the Adjudications and Operations Committee.
6.4 Individual remedial programs
As part of the Public Service Performance Management (PSPM) program for public service employees (RPD members), performance reviews are a continuous discussion throughout the year, during which both good performance and issues are addressed. As part of Performance Management (PM), learning and development plans, and in some cases performance improvement plans (individual remedial programs), are established.
The Strategic Human Resources Management (SHRM) Directorate includes
PM, Talent Management (TM), and Learning. The
PM/TM team helps with all aspects of
PM and
TM and, in collaboration with the learning team, supports managers in developing learning and development plans. Official languages, succession planning, and training are other components related to
PM/TM.
A learning and development plan must be developed for each public service employee and includes both the employee’s career goals and aspirations, as well as planned activities that support the achievement of the employee’s established work objectives, expected behaviours and continuous development.
A performance improvement plan must be initiated when employee performance does not meet expectations, or any time the manager determines that a performance issue needs to be documented. A performance improvement plan must include:
- Specific areas for improvement;
- Actions that will be taken by the employee to address the identified areas needing improvement;
- Description of the support that will be provided by the manager to the employee to improve performance; and
- A timeline and milestones against which to measure the employee’s progress.
The performance improvement plan is a means of addressing individual quality issues. It is usually decided by the member’s immediate manager and could include a range of activities, such as specific professional development training courses, in-house training, mentorship, or returning to New Member Training. Remedial programs reactively address key issues in individual member performance to remediate member specific quality issues.
7.0 Stakeholder engagement
The
IRB’s external stakeholders provide on the ground perspectives of those who appear before the Board, enhancing the development and review of our initiatives and processes. Sustained and proactive stakeholder engagement helps keep the
IRB accountable, responsive and innovative. Additionally, internal stakeholders and personnel, including member cohorts, Legal Services and Tribunal Services, provide feedback on an ongoing basis to support continuous program improvements, as well as improvements to adjudicative and other support tools. As such, stakeholder input and perspectives are sought during the various stages defined in the Quality Assurance Framework. This approach helps strengthen the development, review and monitoring of policies, initiatives and service standards programs.
The Policy, Engagement and Outreach Division is responsible for external stakeholder engagement. To this end, the
IRB leverages its already existing stakeholder networks in order to facilitate and maximize its consultations on the various elements of its quality programs. These networks include:
-
National stakeholders: The
IRB has an established consultative committee (IRBCC) which is the Board’s primary stakeholder channel through which to consult and inform on initiatives and procedures. The
IRBCC meets at least twice annually as well as on an ad hoc basis to consult on specific initiatives. The
IRBCC’s member organizations consist of counsel associations (lawyers and immigration consultants), Ontario’s provincial legal aid program and refugee advocate organizations.
-
Regional stakeholders: Each division also maintains its established regional consultative committees with its local stakeholder groups. Its members are often associated with national stakeholder organizations and are especially helpful when engaging on operational matters where regional differences should be considered.
Additionally, in developing and monitoring its quality programs, the
IRB engages with portfolio partners and subject matter experts (SMEs):
-
Portfolio partners: The
IRB frequently consults its federal immigration and refugee portfolio organizations – Immigration, Refugees and Citizenship Canada and the Canada Border Services Agency – to provide input on operational matters. This is done with the clear understanding of the
IRB’s institutional and adjudicative independence.
-
SMEs: The
IRB frequently consults with SMEs, which include counsel, NGOs and academics who have specialized knowledge and leading-edge experience on specific topics. With regard to the Board’s quality measures, SMEs can provide input on considerations outside the Board’s scope of expertise. SME input helps ensure that the Board’s approaches are informed, respected and cutting-edge.
Comprehensive and sustained external stakeholder engagement helps strengthen the
IRB’s quality programs while also facilitating early buy-in for new initiatives.
From an internal perspective, continuous improvement relies on frequent engagement of members and staff, as well as collaborative relationships with Legal Services and Tribunal Services. This document highlights many of the mechanisms used by the
IRB to ensure that engagement is ongoing and feedback is received and incorporated. Examples include the use of reasons review by Legal Services, Adjudicative Consistency Groups, Performance Management, quality centres and internal evaluations.