Guide to ASQA Audit Process

For about the last two years, ASQA has dramatically changed the way that it undertakes their audits of providers. They now call it a “performance assessment” which in my view is just an extension of woke madness where we try and soften things up to make people feel better about the process. Let’s just be real, it is an audit, the NVR Act calls it an “Audit”, it is an audit.

The regulator explains its new approach to “performance assessment” at its website. I have to say that the information on the process is so generic and high level that it really is not useful. What it doesn’t tell you is what evidence will be requested, when it is requested, what questions will be asked and what other information they are accessing on your operation behind the scenes. Since the commencement of the new process, I have been involved in supporting clients through the audit process from helping clients prepare and engage in the audit, attending audit meetings with ASQA, respond to audit findings, and generally providing advice on the evidence being presented. My overall impression is very positive. Having been involved in VET compliance and quality auditing since 2007, I have just about seen it all. I think the new approach is very good. It is generally a very friendly and relaxed process. The evidence is requested in trenches, so it spreads the workload over a longer period which makes it more manageable for the RTO. There are a series of meetings (videoconference) throughout the process to allow the auditor to better understand the RTO operation. The whole audit is conducted remotely which is great. The timelines for requesting evidence are generous (usually). Overall, I really like it and congratulate ASQA on the model. I would observe that the auditors all seem much younger these days!

Ok, before you conclude that this all sounds wonderful and maybe ASQA are more flexible or there is less scrutiny, stop right there! Whilst it is more friendly and engaging, I think the level of scrutiny is now much more complex to what it was back in, say 2017. This is particularly true for the team that handle the renewal of registration audits. They don’t just look at one piece of evidence in isolation. They connect the lines between items of evidence and how it links with and informs all other aspects of your operation and compliance. I love it. It is a real systemic focus and reinforces the need for a systems approach to managing the RTO and governance. This is important, so if you are fading right now, snap back, and focus on the next point. I just want to give you an example.

Let’s say that the auditor reviews your training and assessment strategy, and they notice that assessment methods allocated to a unit seem to be very “written response based” and there is no obvious practical assessment. This tweaks their interest in the assessment for that unit. So, they include the unit in the sample. They review the assessment tools and confirm that the assessment is not valid. They review the student assessment records and find that there is insufficient valid performance evidence and the student’s responses have been plagiarised from the textbook. They review your assessment validation arrangements and discover that your validation process is not identifying these problems. They cross-check this with your continuous improvement records and see insufficient improvement actions on assessment systems and poor follow through on continuous improvement actions. They link the invalid assessment with you AQF certificate issuance records and confirm that you have issued competency without undertaking adequate assessment, yadda, yadda, yadda. We now have non-compliance across clauses 1.1, 1.8, 1.9, 2.2, 3.1, etc. The key point here is, it is all connected. Some auditors are better at this than other. But, overall it is a very detailed process. ASQA may not be doing as many audits as compared to the past, but the audits they are undertaking are very thorough. I tell you this so you can take note and get your house in order well in advance of a future audit.

Preparing for audit

Do we need to prepare for an audit or should we always be ready? ASQA have a belief that you should always be fully compliant and “preparing for a performance assessment (audit) is not necessary”. They actually say this on their website! It comes from a mindset that recognises that, if you have established your compliance operating system and are diligently implementing this and monitoring the delivery of services closely, then why shouldn’t you be compliant? Those that may have read our article on a systems approach to RTO self-assurance, may recall that I promote a similar concept that “being self-assured means that we can have confidence in the whole business with quality service delivery as the primary objective and compliance is achieved as a consequence”. I do strongly believe this, but I also need to work with clients in the real world. Do we need to systematically prepare for the audit, yes!

In the next section of this article, I will explain what you can expect during a renewal of registration audit with ASQA, what evidence they will request, what questions they may ask, etc. This information is based on our involvement in many audits under this new process where we see slightly different approaches being taken by different auditors based on the risk assessment of the provider. The information I am presenting below is at the higher risk level so if you prepare according to this, you will be going ok.

ASQA undertake the audit according to the following standard process:

1. The Initial contact

The auditor will make contact generally by phone initially to introduce themselves and organise a time for the opening meeting. They will explain the process for issuing a notice of performance assessment and the request for the initial information. If they are unable to reach you by phone, you can expect an email. ASQA get really annoyed when they cannot get in contact with you so it is a good idea to add their domain to your safe sender list and make sure your contact details in ASQAnet are always up to date.

2. Notice of Performance Assessment and Request for Delivery Data

The following day (generally), they will send a notice of performance assessment via email nominating a time for the opening meeting and an attached request for information. The request for information includes the Delivery data request as outlined below. This initial information request is usually required within a week or so and a few days before the opening meeting.

Example Delivery Data Request

1. Using the Delivery Data and Student Survey Data template:

- include a summary of enrolments and completions for the previous 12 months using the ‘Delivery Data Summary’ tab
- include details for students who have enrolled and/or completed any training product in the past 12 months using the ‘Student Survey Data’ tab. This is to include student name, email address and mobile phone number, the code and name of the training product they are enrolled in and their enrolment and completion date
- include a detailed listing of unit of competency enrolments and completions for the previous 6 months using the ‘Enrolment and completion data’ tab. Please do not substitute or provide your data in a different format

2. Provide the following information relevant to your organisation:

- summary of provider and management structure including a copy of your organisation chart (including all staff names, their roles and employment status)
- suburb and state of all delivery locations
- core clients/target groups for each training product
- training revenue (funded, fee for service, etc.)
- the location (address) where your student records are retained
- if student records are retained electronically or in hardcopy
- list of all trainers and assessors and the training products they deliver for all items on your scope of registration
- whether RPL has been conducted in the last 12 months

3. Provide the following information relevant to your third-party arrangements (if applicable):

- a list of third parties and the services they provide on your behalf

3. The Opening meeting

Following their review of the delivery data request, there will be an opening meeting for the audit. This meeting is just an introduction to explain the legislative requirement that underpins the audit, to explain the audit process, the possible outcomes and relevant regulatory implications. The auditor may ask some general questions based on the initial information that you have already provided. These questions can vary greatly but typically they relate to the delivery data summary, access to records, third party arrangements, delivery locations, clarify funding or fee payment sources, etc.

4. Request for Training and Assessment Documentation

Following the opening meeting, you will receive another email this time notifying you of the scope of the audit to be undertaken. It will nominate a time and date for the management meeting and will also include the Training and Assessment Documentation Request as detailed below. You will usually get a couple of weeks to supply the requested training and assessment documentation. The management meeting is usually scheduled a couple of weeks after they have received the training and assessment documentation.

Example Training and Assessment Documentation Request

1. Provide the training and assessment strategy documents and supporting documents for the nominated sample training products. For example:

- policies and procedures that support you to ensure that the training and assessment delivered aligns with the requirements of the standards
- details for the delivery of training and assessment for each training product. For example:
  - where the delivery/assessment will occur
  - topics delivered in each session and the time frames for each session
  - relevant resources, equipment, etc. required for each session
  - assessment task and due date of assessment
  - when work placement will occur
  - equipment register/ resources checklist.
  - timetables for scheduled classes
- any documents or Memorandum of Understanding (MoU) that reflect workplace agreements.

2. Provide evidence of industry engagement within the last 12 months. For example:

- policies and procedures relating to industry engagement.
- evidence of industry engagement such as surveys, correspondence and/or meeting minutes.

3. Provide evidence of assessment validation arrangements. For example:

- policies and procedures relating to validation.
- validation plan/ schedule if not included in the policy/procedure.

4. Provide evidence of your assessment system including for the nominated sample training products.

- student assessment instructions
- assessor assessment instructions
- all materials used to make a judgment of competency for the students. For example:
  - assessment tools/material
  - RPL assessment tools
  - assessor marking guide
  - any other supporting assessment documents such as third-party reports or work placement records (if applicable)
  - assessment mapping tool (if applicable)

5. Provide evidence of trainer and assessor records that describe how trainers and assessors meet the requirements in relation to the units of competency they deliver. For example:

- policies and procedures relating to professional development of trainers and assessors
- training and assessor matrix (if applicable) listing each trainer/assessors competencies at unit of competency level against all training products they deliver and assess
- copies of vocational qualifications/certificates
- copies of trainer/assessor qualification/s specific to training and assessment
- evidence of vocational currency
- professional development records relevant to:
  - the training being delivered and assessed, with supporting evidence
  - knowledge and practice in the vocational industry and competency-based training and assessment.

6. Provide evidence of marketing and enrolment documents for the nominated sample training products. For example:

- policies and procedures (internal and for third parties if applicable) that support your organisation to ensure the accuracy of marketing and enrolment documents
- marketing material other than website
- course flyers
- student handbook (if applicable)
- fee information
- entry requirements (if applicable)
- course details
- items to be provided by the students (if applicable)
- application/enrolment forms template (including interview templates)
- entry/ LLN test template and marking guides
- any marketing or enrolment materials used by third parties if different to those provided above.

7. Provide evidence of Governance documents. For example:

- policies and procedures relating to organisational governance including complaints and continuous improvement
- continuous improvement register
- evidence of continuous improvement such as analysis of feedback and implementation.
- complaints and appeals register
- certificate issuance register

8. Provide evidence of third-party documents:

- a copy of current Third-Party agreements for nominated sample third parties
- evidence to confirm monitoring of these third parties as per the agreement/s has been undertaken for the following third parties:

9. Provide evidence of the following CRICOS arrangements:

- documented policy and procedure for assessing English language proficiency, educational qualifications and/or work experience of students
- information provided to students at the orientation program
- information provided to students on progress and attendance requirements
- policies and procedures for monitoring and managing progress and attendance
- pre-enrolment information for students, for example, a Student Handbook/Prospectus
- student written agreement template
- letter of offer template
- evidence of support services available to students
- name/s of the designated student support personnel
- the name of the official point of contact for students

5. The Management Meeting

The management meeting can often be scheduled over a two to three hour timeframe and will involve your management team and high managerial agents. I would strongly suggest that the owners of the business participate. You don’t necessarily need to have trainers involved in this meeting as the auditor is likely to want to interview a sample of the trainers separately. Keep in mind that by this stage they have reviewed all of the evidence that you have provided so far including the delivery activity summary, organisation overview and all of the documents outlined in the training and assessment documents detailed above. They have already administered a survey to your students and received feedback in response to the survey questions, they have looked at Google reviews and any other available information on your business that is online including social media. If you have a funding contract, they may have received information about your contract performance from the relevant funding authority. If you are a CRICOS provider, they have fully interrogated PRISMS to review your management of COEs, capacity at each delivery site and third party education agents. The point is, they turn up for this meeting prepared and have completed not only a review of the evidence you have provided but have undertaken a comprehensive market scan and have engaged with your students to gather feedback about what they think about your delivery of services. They will have lots of notes already recorded based on all this research and lots of questions ready to ask.

It is not uncommon that there will be two auditors in this meeting with one leading and one assisting and in some meetings that I have participated in there has been a person in the background taking minutes. They seem to do this on higher risk audits. I suggest you do the same. Whilst you are not allowed to record the meeting, you can have a person present who can record detailed notes on everything that is said. I have observed auditors say the dumbest things, behave inappropriately, deliberately try and trick the client up with confusing questioning, bully the client through intimidation and commonly make personal, subjective observations/findings that they cannot justify according to what the standards or the legislation actually says and requires. We recently had an auditor that made a finding that directly contradicts the ASQA published “Guidance for Providers” on the resourcing requirements for applicants seeking initial registration or change to scope of registration. If I see any of this type of behaviour happen during an audit, I call it out and hold the auditor accountable. Don’t let this behaviour just slide by with them thinking that you either acknowledge or agree with their observations, because if you do; that is the way it will be written up in the report. I say to clients from time to time that “ASQA is not your friend”. I know, this sounds harsh, but when I hear a client tell me they have a good “relationship” with ASQA, it is just bollocks. ASQA is a statutory authority, and it does not matter who you are, if you are not doing it right and do not rectify, they will suspend your registration without the slightest of remorse. If you have any notion that you have a good “relationship” with ASQA or think they hold you in high regard, forget this, it is fanciful. As lovely and friendly as some of the auditors are, they are a government agency that should be held accountable. If you don’t speak up and defend your operation and compliance, they will roll over the top of you, so speak up and make sure you have someone taking detailed notes on everything that is said.

The questions that you are asked in this meeting will relate to the evidence that you have already provided and any points of clarification that the auditor has. They will also ask you a multitude of questions about how your RTO operates with a general focus on governance. The expectation of governance in this audit is huge. This is important so before we jump into some of the sample questions, I just want to highlight the importance of governance. ASQA expect that you have an established system of policy and procedures that include appropriate quality assurance arrangements. They expect the CEO to be fully engaged in the implementation and monitoring of the RTO operation to ensure compliance. The CEO and senior management need to have a detailed knowledge of the entire operation and of the compliance requirements. Ignorance or apathy are not an option if you want to hang on to your RTO registration.

To understand how governance is implemented, it is useful to look at just one example of how this is applied. Let’s take assessment. After you have selected a unit of competency based on your industry consultation, the assessment tool should be designed and developed and then validated in a process of pre assessment validation before it is implemented. There should be records of this and the industry consultation that informed the assessment design. When the assessment is implemented, there should be process of introducing the assessment to trainers and assessors to make sure they are all fully informed about how the assessment should be applied. There should be evidence of this. Once the assessment has been implemented, there should be a process of conducting assessment quality control to ensure that only adequate assessment is being accepted to inform student results. There should be a process and evidence of this. Assessment should be subject to assessment validation which should identify where the assessment is not fit for purpose or compliant. There should be evidence of this. Remember that findings that resulted from assessment validation relate only to a sample of the units of competency and therefore these findings should be applied not only to the sample units but to all of the units of competency relating to that course and potentially even wider. There should be evidence of this. There should be a clear connection between processes such as assessment quality control, pre assessment validation, post assessment validation with your continuous improvement process. If there is a breakdown in any of these processes, then it is the CEO’s fault because they should have put arrangements in place to monitor the implementation of these arrangements to make sure they were working. I know this sounds harsh but at the end of the day, the CEO is the person responsible. That’s what the legislation says. When we do root cause analysis following one of these major non-compliances, it always results in identifying that there was a lack of oversight and management to ensure the implementation of appropriate arrangements. Sure, you can blame middle management because maybe they weren’t doing the job that they were being paid to do but at the end of the day, the CEO should have identified this problem earlier and done something about it. Governance, it is a big focus in these audits, and it needs to be your focus.

Here are just a sample of questions that you may be asked:

How do management ensure that training assessment is meeting the needs of the training package?
Can you explain how you apply version control as part of your document management process?
Can you talk me through the process for developing and approving your training and assessment strategies?
How do you ensure that your training and assessment being delivered (practices) aligns with your training and assessment strategies?
How do you determine that the amount of training you are providing is sufficient for the target learner?
Can you talk me through how you involve industry in the development of your training and assessment strategies?
Can you talk me through the process for recruiting your trainers and assessors.
How do you verify the competency and currency of your trainers and assessors?
How do you ensure that trainers maintain their competency and currency?
How do you determine the resource requirements in support of a course?
How do you determine if a course has sufficient resourcing?
What process do you apply to validate and customise your learning and assessment resources?
How do management ensure that the assessment being administered is compliant and supports the issuance of AQF certificates.
Can you talk me through your arrangements to verify the adequacy assessment before the issuance of competency.
How do you determine if students require support services?
How do you monitor the need for support services during the student’s enrollment?
What controls do you have in place to verify the compliance of marketing materials before these are relied on?
Can you talk me through your arrangements to apply continuous improvement to your training and assessment.
What data and information do you collect to monitor the quality and compliance of training and assessment?
Can you talk me through the process for commencing and ceasing third party arrangements.
What monitoring arrangements are you applying to monitor the services being delivered by third parties on your behalf?

These questions are just a sample of what I have observed during these management meetings. One important point I would make is this, they are taking notes of everything you say. I have lost count of the number of times I have seen in an audit report where the auditor observes that the policy document said one thing and the compliance manager, or the CEO explained a different process during the management meeting. That’s why before the management meeting I would encourage you to make sure you are familiar with your arrangements and compare these with your practices. Allocate some time to prepare how you are going to respond to these types of questions.

6. Request for Practice Documentation

Following the management meeting, you will receive the final request for evidence which is the Practice Documentation Request detailed below. The final request focuses on evidence of your implementation of arrangements. As an example, in the previous request for information they requested evidence of your assessment validation procedure and plan. In this final request they want to see evidence of your implementation of assessment validation such as your validation records. In the previous request, they requested evidence of a sample of assessment tools. In this final request they want to see evidence of your implementation of these assessments in a sample of student assessment records. You are usually provided a couple of weeks to gather and submit this evidence.

Example Practice Documentation Request

1. Provide evidence of the facilities/equipment in support of the delivery of the nominated sample training products.

- list of physical resources/equipment (if applicable)
- add photos/videos of relevant facilities and equipment (if applicable)

2. Provide evidence of the learning resources for the nominated sample training products.

- a breakdown of the learning resources, including a resource mapping tool (if available) for each unit of competency. For example:
  - student workbook/learning resource (hardcopy or online)
  - trainer notes
  - PowerPoints
  - handouts.
- electronic copy of resources or details of purchased resources including version number where resources cannot be sent electronically
- login details for assessors to access online platforms (if applicable)

3. Provide evidence of the student assessment records for nominated sample students.

- completed student assessment records with all assessment evidence relied on to form the assessor’s judgement of student competency (including RPL, if applicable) and the corresponding assessment tools and marking guide/s (if different to those described above)
- written instructions on how to access any student files held on an online platform including login details (if applicable)
- completed work placement agreements (if applicable)
- AQF certification (if applicable)

4. Provide evidence of the student enrolment records for nominated sampled students including:

- Enrolment application forms
- entry/ LLN tests
- correspondence between you and the student
- agreement to terms and conditions of enrolment

5. Provide evidence of assessment validation records. For example:

- evidence of validation. For example:
  - meeting minutes
  - validation forms
  - any other assessment instruments used in validation activities.

6. Provide the evidence of the implementation of CRICOS arrangements, including:

- Records of attendance for the sampled students detailed below (last 6 months)
- Records of monitoring and managing progress for the sampled students detailed below
- Letter of offer for the sampled students detailed below
- Student written agreement for the sampled students detailed below
- Evidence of support services for the sampled students detailed below (where applicable)
- Evidence of fee payments for the sampled students detailed below
- Evidence of SCVs issued for the sampled students detailed below (where applicable)

I should just make the point that there can be alternatives to the above practice documentation request where the auditor may also request evidence on potential gaps in compliance they have identified from the review of the training and assessment documentation. An example of this might include nominating a particular trainer where vocational competency has not been clearly demonstrated and requesting additional evidence relating to a sample unit to verify vocational competency or currency. This has become more common. Another example might include where they request evidence of your rationale for the amount of training being provided in a particular course where they may have identified that the amount of training being delivered is insufficient. These types of targeted requests are actually a good thing for the training provider. It says that the auditor has identified a potential issue and they are requesting very specific evidence to allow the provider to address the issue before it becomes a non-compliance at the closing meeting. This does not always happen and seems to be auditor specific and from my perspective is an indication of the auditor’s level of attention to detail and good audit practice. Keep an eye out for these very targeted requests.

7. The Closing meeting

Following the auditor’s review of this final evidence, a closing meeting is scheduled where the auditor will talk through any significant findings and explain the process from this point. This can include identifying areas of non-compliance and identifying minor discrepancies which are communicated so that the RTO can address these. If there are non-compliances and following the issuance of an audit report, the audit will be handed over to a different compliance team who will administer any further follow up action which may include providing further evidence to address non-compliance, conducting a review of decision, entering into an agreement to rectify, or providing a response prior to ASQA making a decision such as suspension or cancelation of registration. Make sure that you take detailed notes during the closing meeting because it is often the case that preliminary findings which are communicated may vary greatly with what is recorded in the audit report.

Conclusion

Too often I have RTOs that contact me once they have received a report with one of these serious non-compliance outcomes. Sometimes these might be previous clients that we have not heard from in a while, sometimes these are completely new clients that have been referred to us. Whoever they are, I always wish they had contacted me 12 months ago so we could have helped them to properly prepare for the audit to prevent these issues from ever happening. I recommend that you start your preparation at least 18 months out from the date your registration is due to end. Remember that you need to submit your application to renew the RTO registration 90 days before the registration end date. It is critical not to miss this deadline. The application is available to submit on ASQAnet six months prior to the registration end date. I encourage you to use the information in this article to prepare your documentation and undertake your own self-assessment to ensure that you have the evidence in your practices to back up your strategies. Organise this information into a shared folder and when the national regulator requests the information, you should only need to update the information based on the request. Remember that the request that you receive may vary to the above examples based on the risk assessment of your RTO. We still get clients that have their RTO registration approved without an audit, but this is definitely not something that I would plan on happening.

Good training,

Joe Newbery

Published: 12th April 2023

Back to Articles