Speaking ASQAneese – Part 1
Have you recently found yourself sitting on a Teams meeting with an ASQA Auditor and they are going through a series of questions that you are just dumbfounded by? Its like, they ask the question, and you think to yourself – She is definitely speaking English but I have no idea what she just asked me. Ok, you are not alone. Don’t worry, it is not you. You are normal. You are intelligent. You are not stupid.
In this three part article series, I want to try and explain what these questions are asking. If you haven’t yet been exposed to the famous ASQA inquisition called the “Management Meeting”, this article will be helpful to assist you in understanding the actual question behind the question.
How did ASQA come up with these questions?
I understand in November 2021 after the end of the second lockdown, ASQA called in its top ten bureaucratic operatives from around the country to a secret meeting in the Dungeon of the Sydney Elizabeth Street office. After getting past security which apparently involved a secret handshake, only ASQAneese was allowed to be spoken to counter any attempted electronic eavesdropping. Apparently, there were also two special invited guests who were Senior Bureaucratic Tacticians from the planet Zorgon. Together, this crack team of bureaucrats would be known as the “ASQA Dozen”. The Dozen (as they are now referred) set about unpacking the standards and crafting questions that whilst using English as the basis for the words in the question, the questions would masterfully string together these words in a way that would be sure to confuse even the most experienced Commonwealth Department Secretary. The Dozen worked tirelessly and after two days and two nights they emerged on the third day to survey the body of questions they had created, and the Dozen saw that it was good. It was a master stroke. The questions would both bamboozle and systematically inflict the feeling of stupidity on the recipient. With their work completed and after a couple of Chardonnays at the Fix Wine Bar, the Tacticians beamed back to Zorgon and the ASQA Operatives dispersed to their home kingdoms. Job done!
Considerations when responding to questions
I think it’s important to note, these questions are being asked in the context of some type of performance monitoring or audit. ASQA has access to all sorts of other information that relate to the questions they are asking. Primarily this includes:
- Information they have collected on you externally such as publicly available information from the internet, information sharing from other agencies, survey responses from your students.
- Information you provide them on your strategies and arrangements such as your policies and procedures, training and assessment strategies, et cetera.
- Information you provide them on your implemented practices such student enrolment and assessment records, continuous improvement records, validation records, record of complaint handling, et cetera.
This next point is probably the key take home point from this article so please take note. When they ask you a question like “Can you explain how you ensure your marketing material is accurate and reflects your registration obligations?”, you need to keep in mind that they have already read your marketing policy and documented arrangements, they have already looked at a sample of your marketing material that you have provided and also your marketing material that is publicly available such as your website. They have already formed a view about the compliance of your marketing. So, you can see that the question is essentially a bear trap. They are basically inviting you to reinforce and acknowledge what they already know.
I see clients just start blurting out information to the auditor without thinking about what their policy and procedure says and in doing so contradicting their own arrangements. This sends a very damaging message to the national regulator. It basically says that the management have really no idea what their own policy and arrangements are and therefore they are not implementing these arrangements and have very poor governance. I say to clients regularly that, when you answer these questions you not only need to understand your state of compliance in relation to the question, but you also need to understand your own policy and arrangements. When you answer the question, you need to not contradict your own arrangements and you also need to fully understand where your current arrangements are either compliant or need improvement. If you have already verified that your practices are compliant and you have well documented arrangements, then you can talk confidently about these arrangements and their implementation which is just reinforcing what the auditor already understands. If you know that your practices are non-compliant, then you can acknowledge this in your answer. I suppose the key point is, you need to have a knowledge of your own arrangements and your own state of compliance and be prepared to answer the question in a way that doesn’t contradict yourself or make your organisation look stupid.
I have lost count of the number of times I read a different version of the following statement in audit reports:
“Review of the organisations marketing material identifies that the marketing and advertising of services is not consistent with the services being delivered. The provider’s marketing policy and procedure indicates that a quality check will be undertaken on all marketing material before it is released to the market using a marketing checklist and that the CEO is required to approve all marketing material. Noting this, at the management meeting, the CEO indicated that the training manager is responsible for the release of marketing material which is updated regularly. The CEO was not familiar with any marketing checklist that was in use and the organisation was not able to provide any example of a marketing checklist that it had completed. The organisation’s practices are not consistent with its own policy and do not accurately represent the services it provides and the training products on its scope of registration”.
I have read a similar statement for just about every clause in the standard. What does it say about the RTO? It says that the organisation has a policy that they a not familiar with and do not implement. It says that the CEO is not taking an active role in the governance of the organisation’s compliance. It says that the organisation’s marketing material is non-compliant and it has insufficient arrangements to comply with the relevant clause (4.1). Now, you might think I am being a little harsh but seriously, it is better to hear this from me than to get it in an audit report. I am sure that anyone that has been recently through a renewal of registration audit will reinforce what I’m saying here.
Key points to take away:
- Understand your current state of compliance and what you are currently doing to improve this.
- Have a detailed understanding of your current documented policy and procedures.
- Review and rehearse your answer to these questions in a way that does not contradict your documented policies and procedures and acknowledges your current state of compliance.
- Work hard on making sure that your arrangements are compliant and are in alignment with your documented policies and procedures.
I might just finish with one more sneaky point before we get into the questions. Be careful what you say in one of these performance monitoring audits. Everything you say is being recorded by a note taker (usually the auditor or an assistant) and will be used in the subsequent audit report. Also, if you say that something has occurred that has some type of record associated to it, be prepared that the auditor will say “fantastic, can you provide me a copy of that”. Sometimes I sit there in a performance monitoring management meeting thinking to myself “far out, please stop talking!”. Sometimes, clients feel very comfortable because the auditor is so nice, and they get on a roll and just start telling their entire story. They start telling the auditor all sorts of things because they think it is impressing the auditor but in reality, what it’s doing is opening lots of cans of worms. Just answer the question as simply as possible and don’t mention anything that you cannot verify in a record. Because they will ask for it!
Also, sometimes you are better off providing a very top level general response to a question and then let the auditor come back with another question drilling down to something more specific. I will touch on this in the article but I do think this is pretty important. Many of these questions are really big questions. Many are simply too big to answer one response. If you think about the marketing question example that I provided in an earlier paragraph, I might respond to that question by simply saying that “Our marketing material is prepared in draft and then checked using a marketing compliance checklist before being passed to the CEO for approval’. That’s it. Answer the question with as few words as possible. Put the audit evidence gathering obligation back on the auditor to come up with a more specific question to which you can then respond. OK, let’s get into it.
Interpreting questions in ASQAneese
I have presented the questions below according to the standard or area of the standards that they apply to. These questions have come from a range of sources including being asked during performance assessment management meetings, issued to clients in information requests, issued as part of a strategic review, et cetera.
Training and Assessment Strategy Questions
Question: How do management ensure that training assessment is meeting the needs of the training package?
Interpretation: This question is really asking what arrangements you apply in the development of your training and assessment strategies to ensure these are aligned with the training package. By the “needs of the training package” we are referring to qualification framework requirements such as the options for the selection of elective units, entry requirements, mandatory work placement, pre-requisite requirements, unit of competency requirements such as assessment requirements, et cetera. Your answer might refer to how you identify the initial need for the training, how you identify the target learner and their needs, the process to consult with industry, the planning of resources connected with delivering the training and assessment, the final production, review and approval of the training and assessment strategy. You could talk about how you analyse the unit of competency requirements to make sure that these are all considered during the design process. Some of our clients send their training and assessment strategies out for an external review and approval. So, the client can talk about this external review and provide evidence if requested.
To be honest, I find the part of the question that clients get tripped up on is the final part about “training package” requirements. It’s such a big topic that the client really does not know where to start. That is the problem with many of these questions. They have made the questions too big that would otherwise encourage a more immediate and succinct answer from the provider. For someone like me that lives and breathes this stuff every day, it’s easy-peasy but for the everyday training provider that is distracted by developing their business and providing learner support, the immediate answer to these questions is not front of mind. It is almost like they are deliberately complex. Sorry, I will try not to rant on in response to every question.
Question: Can you explain how you apply version control as part of your document management process?
Interpretation: This is a pretty reasonable question, and I don’t see anyone having problems with this one. It’s almost like one of the humans from the Dozen came up with this question, maybe from Tasmania. Tasmanians are such nice people. I should just be clear and make the point that there is no requirement in the standards for RTOs strictly for version control. I would love to see a non-compliance on version control go to the AAT and see how that plays out for ASQA. Notwithstanding that, obviously it’s good to have a system with version control. Generally, we denote minor changes using a decimal point such as 1.2, 1.3, et cetera and when we have a major change we might move to the next version completely such as 2.0. Having and maintaining a document register is also recommended.
This question is usually born from something that they have already identified, and they are trying to trip you up into admitting that you don’t apply a system of version control. As an example, maybe they have had a look at your continuous improvement register and identified that there was a recommended improvement regarding one of your training and assessment strategies. They compare this with the training and assessment strategy for that course that you have already provided, and it happens to be in version 1.0. You can see that the question is motivated from the fact that they realise there should have been a version change when the improvement was implemented, and they are trying to find out what system you have in place and determine why the version change didn’t happen. It all links back to governance. You might explain your version control arrangements in response to the question and then they ask “OK, thanks for explaining that. I was just to be confused about your version control arrangements because I noticed that in your continuous improvement register it identified some required updates to the training and assessment strategy but this training assessment strategy you have provided appears to be version one dated before this continuous improvement item. How do you explain that?” I suppose the moral to the story is that it is best practice to apply some type of formal version control with a document register and when you send documents to the national regulator, particularly make sure that these documents have the correct version number.
Question: Can you talk me through the process for developing and approving your training and assessment strategies?
Interpretation: I know, and you know that most RTOs do not have a documented policy and procedure around the development and approval of a training and assessment strategy. It usually goes something like this, “Hey Peter, can you contact Mary Blogs from XYZ age care facility and have a chat to her about what elective units of competency they are looking for and what their preferred arrangements are for work placement and put together a training and assessment strategy and have that on my desk by Friday please.“. Seriously, ASQA are dreaming. It is interesting to note that the draft RTO standards don’t even mention the requirement for a training and assessment strategy. I wonder if that will continue into the final document. I see some big public providers that do have a well-defined process for this and if you ask them, they will tell you that it absolutely bogs them down in bureaucracy just to get a training and assessment strategy approved. If you don’t have a documented process for the development of a training and assessment strategy, don’t worry because it is not a prescribed requirement within the standards. It only becomes a problem if your training and assessment strategy is non-compliant. Remember that we operate in an outcome based compliance model not a process driven compliance model. It is 100% sufficient for you to verbally describe the process that you apply that may not be documented but is well understood by all concerned and that involves suitable checks and balances to ensure that training and assessment strategies are developed appropriately.
Your answer might simply identify that the process normally begins with consultation with the client or industry to determine their training needs, these training needs are taken into consideration in the development of the first draft which is used for the basis for review and comment and further consultation with the client. After the consultation is taken into consideration and final updates are made to the document, the final document is produced and provided to the CEO for review and approval.
Question: How do you ensure that your training and assessment being delivered aligns with your training and assessment strategies?
Interpretation: This might sound like a straightforward question but, far out, I see clients just get bamboozled by this question. It’s like, they understand the question, but they really don’t know where to start to explain. The question is basically asking about how you monitor the delivery of training and assessment according to what the training and assessment strategy required. What is this talking about? It’s talking about how you monitor the scheduling of training in accordance with the planned course program, the provision of equipment and resources in accordance with the resource plan, the student’s attendance according to the planned scheduled activities, the alignment of learning and assessment resources with the unit requirements, the use of feedback collection and training evaluation to monitor the effectiveness of training, et cetera. If you are not familiar with training evaluation strategies, it would be worth having a read of the Kirkpatrick training evaluation model. This provides a good model for all training providers to ensure that the training being delivered aligns with your training and assessment strategies and is achieving the desired outcomes for the employer.
Question: How do you determine that the amount of training you are providing is sufficient for the target learner?
Interpretation: Seriously, this is a doozy of a question. I am confident that the brothers from the planet Zorgon had a hand in this question. Firstly, to answer this question you really need to be familiar with your training and assessment strategies and the amount of training that you have allocated together with things such as the target learner characteristics particularly any assumed prior knowledge and skills, course entry requirements, existing competency, the efficiency in the mode of delivery, et cetera. If none of these things make sense to you, read this past article the-amount-of-training-part-one. Fundamentally, you determine the amount of training for a course based on taking into considerations the number of learners in the cohort, the availability of resources, the characteristics of the target learner particularly prior knowledge and skills (their entry point), the training to be provided in relation to the number of units of competency, the unit complexity, the type of skills to be trained and practiced (practical / cognitive) sufficient to prepare the learner for assessment. Remember that learners need to develop and practice their skills prior to assessment. The primary focus is demonstrating through the provision of a detailed course program that learners will have sufficient opportunity to practice and develop their skills prior to assessment. If the auditor understands the fundamental of course design they will have undertaken their own time / space analysis based on the resources, the time allocated and the number of learners to determine if there is enough time allocated to skill development. Here is the thing, let’s say you have a learning activity that is allocated 30 minutes per person. You have enough resources for two learners to do the activity at the same time. You have 12 learners, so you allocate 6 hours to the activity. It makes sense that if you are doing this for learners to practice you would allow for practice on more than one occasion (so 12, 18, 24 hours). You see the point. You need to do a time space analysis based on all the considerations to determine if the learner is getting enough time to practice and develop their skills. The auditor has already done this and so when they ask the question, it is basically a loaded question. If you know the detail at this level, then you will have no problem answering this question. Also just keep in mind your arrangements for monitoring the sufficiency of the amount of training. You monitor the sufficiency of the amount of training by monitoring the learner participation in training, instances of reassessment, attendance, feedback from trainers, feedback from learners, ultimately feedback from employers.
This is another big question. One trick that I eluded to earlier is how to provide an initial answer to these big questions and then let the auditor ask another more specific question. As an example, in response to this question you might simply provide the answer “We take into consideration the needs of the target learners and also the requirements for the qualification to allocate a sufficient amount of training to allow the learners to develop their skills prior to assessment”. That’s it. What you are doing with a response like this is answering the question but now reverting it back to the auditor. They need to have properly listened to your answer (many don’t because they are too busy taking notes), interpret the answer and then ask the next logical question which generally will be narrower in scope such as something like, “OK, great can you tell me about what you are considering in the target learner when allocating the amount of training” or they may simply say “Ok, that is great” and move on because literally they don’t know what to ask next.
Remember that the ultimate consideration in determining how much training to allocate in a course is to allow sufficient time for the learner to practice and develop their skills with feedback from the trainer prior to being expected to demonstrate competency in assessment. The amount of time you allocate to this can be influenced by the pre-existing knowledge and skills of the target learner. If you look at your course and you cannot identify sufficient opportunities where learners can practice and develop their skills prior to assessment, then you probably have a problem.
Resourcing Questions
Question: How do you determine the resource requirements in support of a course?
Interpretation: This is a reasonably straightforward question. The only partially complex part of the question is the word “resource”. Some clients would immediately start talking about the learning and assessment documents they purchased. Resources involve much more than just those types of documents. We are talking about the equipment, the facilities, the number of trainers, the consumables together with things such as the number of learners in the cohort and the time allocated for training and assessment. Obviously of primary importance is the resource requirements of units of competency. These may be specified within the assessment conditions or be apparent as a requirement of the performance evidence requirement. In many units of competency, the assessment conditions are just hopeless so you really need to look at the tasks that the learners are performing and get a subject matter expert involved to determine what equipment and resources would be needed to perform these tasks. The immediate answer might be, “We undertake an analysis of the units of competency to identify the equipment and resource requirements and determine these requirements based on the number of learners on the course and the time allocated.”. The next question is probably going to ask for evidence of the identified resources such as an inventory list of resources in support of the course which I would encourage you to have. Don’t forget things such as work placement facilities, verifying on-the-job training resources, access to suitable resources in a simulated work setting including the knowledge infrastructure such as organisational policies and procedures, et cetera.
I suppose the take home point here is that you need to get into the detail. I see some clients wanting to put on scope commercial cookery, but they have no idea about the complexity and detail of that training package in relation to the equipment that is required. A client might want to deliver an online course in leadership and management, but where is the simulated workplace, the scenarios, case studies and organisational context coming from? A client might like to deliver their training at the client’s premises, but what is the equipment that you need to have on site in sufficient quantities and serviceable to support training? Make sure you get into the detail and have fully considered the resources required for the delivery of your courses. Provide the initial response and then respond to any subsequent questions as they come.
Question: How do you determine if a course has sufficient resources?
Interpretation: This is a similar question that we have had previously in regard to the monitoring arrangements that you have in place. How do you determine if a course has sufficient resources? By doing the planning upfront to determine the required amount of resources (see previous question) and monitoring the utilisation of resources to ensure that there is sufficient resources available to trainers and learners throughout the learning and assessment activities. OK, that is your initial response and then the next question is going to be, “how do you monitor the utilisation of resources?”. You might identify that your primary method of determining if the course has sufficient resource is engaging with trainers on a regular basis to determine how the training is going and getting their verbal feedback about sufficiency of the resources available. If you are delivering a course which has a high volume of consumables, you might have a process in place to order and restock these consumables on a regular basis which is worth talking about. If you are delivering on site training at the client’s premises, this is where you would talk about your process to verify (using a checklist) the resources available at the premises and their suitability in support of the training to be delivered. Obviously, as soon as you mentioned the word “checklist” you can be sure as eggs that the auditor will ask for examples of these. The answer to this question will vary based on your mode of delivery. Regardless of this, you just need to think about how you determine if resources are insufficient or low and how would you respond to this. Keep it basic and make sure you have evidence to back up whatever you claim.
Question: What process do you apply to validate and customise your learning and assessment resources?
Interpretation: The question is reasonably plain language and straightforward but, there is a motive behind the question that you need to understand and make sure that you address. The vast majority of training providers use commercial resources. Now, it would be lovely if all of these resources were fully compliant and magically customised for your mode of delivery but, you all know that this is never the case. All these resources are non-compliant to some degree. Taking them straight off the shelf and putting them in front of students and trainers without any validation or customisation is basically a recipe for disaster. You might get away with this for a while but at some stage it is going to catch up with you. This question is seeking to determine what your process is for pre-validation of the resources and how you customise these for your mode of delivery. If you do not do this, then I recommend simply being honest and letting the auditor know this. Maybe you had undertaken a review of a sample resource which you determined to be compliant and aligned with your delivery model and following this initial review you purchased the full package and implemented it without any further review. It is probably not the best response but at least it gives the impression that you did undertake some level of review. Best case, you explain that each resource was subject to a validation of the supplied mapping tool to confirm the coverage of the unit of competency by the assessment tasks. Where changes were required, these were updated in the supplied mapping documents. Maybe you had a sample of the resources externally reviewed with feedback and the outcomes of this external review were implemented across all assessment tools.
The resources were updated and customised for branding and customised for your local mode of delivery which required adjustment for delivery in an on the job training model (just an example). There is no perfect answer for this question other than simply explaining the process that you apply. Just be aware that I have had an auditor request the client provide a copy of the originally supplied commercial resources so they can compare these with the current ones in use to verify the claims made by the client about their customization. It is worth noting that pre assessment validation is not a current requirement within the standards. I agree that it is best practice, and it has been proposed to be included in the draft RTO standards, which I suspect it will.
Industry Engagement Questions
Question: Can you talk me through how you involve industry in the development of your training and assessment strategies?
Interpretation: You can see that these questions are really high minded and many go beyond the requirements of the standards. This question is associated with a requirements from the standards (clause 1.5 and 1.6) which require the training provider to ensure training and assessment strategies are relevant to the needs of industry. ASQA are taking it a step beyond that and suggesting that the industry needs to be “involved” in the development of the training assessment strategies. This is why most clients get stuck on this question. If the auditor had asked “Can you talk me through what strategies, you apply to engage with industry in the development of training and assessment strategies?” you can see how that question directly relates to the standards and most clients could answer that question. Instead, the client is trying to think of an instance where industry was “involved” in the development which generally they would not be. You just need to interpret this question in relation to industry engagement. Simply explain the process that you go through to engage with industry about their training needs and what information you gathered from industry in the development of your training and assessment strategies.
Remember that the standards require you to apply a range of strategies so this might include direct consultation, attending industry networking events, visiting industry work sites, conducting a feedback survey with industry, et cetera. It is good if you can have a couple of examples up your sleeve where your industry engagement has led to specific outcomes that you can point to in existing course offerings. This might include things like adjusting elective units based on a workplace requirement, using industry standard operating procedures to inform how assessment tasks are conducted, introducing industry forms into the learning and assessment process, et cetera. It is good to have these available and make sure that you have evidence to provide because as soon as you offer one of these examples, the auditor will ask you to provide that evidence. So don’t get distracted by the word “involvement”, just stick to the requirement of the standards which requires industry engagement.
Assessment Questions
Question: How do management ensure that the assessment being administered is compliant and supports the issuance of AQF certificates.
Interpretation: This is essentially a governance question. This is one of those questions that if you don’t think about the answer you are going to provide before the question is put to you then it would cause you to pause and try to understand the question better before you responded. Keep in mind that everything that happens within the organisation is the management’s responsibility. So, the question is really asking, what arrangements do you have in place to ensure that your assessment is compliant. Don’t look into it beyond that. When responding to these questions that have a governance focus, you need to be thinking very wide and not narrow in terms of actions that are taken but wide in terms of the systems that you have in place. As an example, you might have a process to undertake pre-assessment validation on assessment tools before they are implemented, you would have an arrangement for post-assessment validation as required by the standards, maybe you have implemented a program of professional development with assessors to moderate their assessment practices, you might have an assessment quality control arrangement that requires assessment to be verified before they are submitted to admin and then checked again before the outcomes are accepted. Lastly you might have a program of external audit to verify the compliance of assessment.
The key point here is you need to have controls in place that ensure any unit of competency issued to a learner is based on valid and compliant assessment. It is important to make sure that whatever arrangements you are explaining in response to the question that you are confident these have been applied an implemented and if requested you can provide evidence of this.
Question: How do you ensure that assessment meets the principles of assessment and the rules of evidence?
Interpretation: To be honest, this question really annoys me. When you read the question out it sounds pretty simple, but the reality is it is a very big question. I just wonder if anyone at ASQA considers this? I also can say with a fair degree of certainty that most of the auditors that I observe during these audits would be flat out actually rattling off the principles of assessment and the rules of evidence let alone having such an intimate knowledge of them as to understand the potential answer. A few weeks back I watched the auditor ask this very question and the client provide an answer something like “we validate our assessments to make sure they are compliant”. I am sitting there thinking to myself, “OK yep but there is obviously a lot more to it than just doing validation”. I am watching the client trying to think of something else to say and I am about to jump in when the auditor simply says “OK great” and moves on. I sat there just thinking to myself, far out, what a joke. Either the auditor just doesn’t understand the question they are asking or they really didn’t care about the answer anyway. I mean, why even ask the question unless you really want an answer that addresses the principles of assessment or the rules of evidence. This is one question that I think needs to go back to the Dozen to be reconsidered.
OK, now with my little rant out of the way, I recommend to clients that you break this question down into two parts:
-
- The principles of assessment primarily relate to the way assessment is conducted or the assessment process. We have designed the assessment based on the unit of competency and workplace requirements to make sure it is valid. We have developed marking guides and benchmarking to make sure it is reliable. We have established detailed assessment procedures and instructions Including arrangements to prevent plagiarism, opportunities for appeal, mechanisms for reasonable adjustment to ensure that the assessment is fair and flexible, et cetera. So, you can see how I have responded to the first part of the question which deals with the principles of assessment.
- The rules of evidence really focus on the quality of the evidence that you are gathering in the assessment process. I would respond by saying that we collect evidence of the students work relevant to the unit of competency to ensure that the evidence is valid. We either observe the student produce this work to ensure its authenticity or we have in place arrangements to authenticate the evidence such as mandatory referencing, authenticity declarations and plagiarism detection. We have designed the assessment to collect sufficient evidence based on the unit of competency requirements on the required number of occasions covering all unit requirements. Lastly, we typically make the assessment decision close to the collection of the assessment evidence is collected either at the time or in the recent past to ensure the evidence is current.
That is the answer that I would provide but again, I really don’t understand the purpose of the question. It’s almost like it is a knowledge assessment. If they are really interested in the way that you are undertaking assessment and have observed something in your assessment tools that they are concerned about, such as, there is no marking guide, you would think that they would ask a much more targeted question such as “After having a look through your assessments, I just wanted to understand a bit more about how you ensure reliability in your assessment decision making. Can you tell me how you ensure reliable assessment”. That is a great question and it’s much more targeted to a particular area of interest in the assessment process and it’s something that the client can respond to. My gut feeling is that the quality of knowledge and understanding of training design and competency based training and assessment in the current flock of auditors is so poor that they have needed to write these questions out for them so that they are simply working off a script like a customer call centre. I guess from your perspective, you just need to make sure that you fully understand the complexities of the principles of assessment and rules of evidence. I would break it down into two parts like I have suggested because I think it makes more sense.
Question: Can you talk me through your arrangements to verify the adequacy assessment before the issuance of competency.
Interpretation: This question is about your assessment quality controls that you have in place that prevent an assessment from being entered into your student management system as competent without first verifying that the assessment is compliant and meets the requirements of the training product. There is a great webinar on assessment quality control on our website if you are interested. The question essentially relates to clause 3.1 which requires that the RTO only issues competency outcomes to learners it has assessed as meeting the requirements of the training product. It is a clause that is essentially linked to clause 1.8 which relates to assessment. It is about the assessment practices employed by your assessors and the internal capability within your organisation to identify where these practices are inadequate to rectify these before issuing the competency. Let’s just consider an example. Let’s say that the student is plagiarising their response to knowledge questions from the learner guide word for word. The assessor is lazy and simply drones their way through the assessment marking process and ticks these activities off as satisfactory without any consideration of the plagiarism that has occurred. Every student record is the same and so it appears that all of the students have all simply transcribed the content from the learner guide into the written response knowledge assessment. This is plagiarism 101 and the organisation needs to prevent this. The assessor did some Mickey Mouse TAE course and takes no pride in their work (can you believe that you are paying this guy $65 per hour!!).
Now, how do you identify this systemic plagiarism before these assessments are passed through to the admin team where the outcomes are updated in the student management system? You need some type of systematic checking or intervention that either checks all these assessment records before they are accepted or at least takes a sampling approach to identify where the assessment is not adequate. Keep in mind that this might need to be conducted in a couple of different ways because, it’s difficult to put this burden on the admin team alone. The admin team do not necessarily have the competency or expertise in assessment to identify when the evidence does not meet the rules of evidence. The admin team can check for things like if the assessment is complete, if the assessment is signed, dated, did the assessor record sufficient comments and observations. These are reasonable things that the admin team can check. But it is probably a bit beyond the expectation that they should be expected to detect plagiarism or to interpret assessment evidence. So, you can see in that situation you would need something else in the mix such as some type of assessment moderation process conducted by someone qualified in assessment that either looks at all of the assessments before they are accepted or at least looks at a sample.
I would also just finish by saying that your systems and processes that enable adequate assessment to occur begin way before the final quality control. If I was answering this question, I would say that our confidence in the adequacy of the assessment began when the assessment was first designed to ensure that it meets the principles of assessment and this was verified through assessment mapping and through pre assessment validation. The qualification was subject to post assessment validation where the opportunities for improvement identified through that process were applied to all the unit assessments within that course. We undertake regular professional development with assessors to ensure that their assessment practices are leading to the collection of evidence in accordance with the rules of evidence. This includes assessment moderation to ensure that all assessors are assessing to the same standards. Lastly, we apply a system of assessment quality control where the assessor themselves needs to complete a checklist to confirm that the assessment has been completed appropriately before this is submitted for the reporting of results. The assessment then goes through a final assessment quality control conducted by the administration team before the results are entered and the record is stored. Yadda yadda yadda. I think what this shows is that there is an entire assessment system that involves people and process that all needs to come together for the assessment to be adequate. When you are responding to this question, you need to look at it from a systems perspective.
Conclusion
I decided to break this article up into three parts primarily because of the number of questions as it otherwise just becomes too big. In part 2 there are about another 22 questions that cover areas of the standards relating to trainers and assessors, continuous improvement, and governance. I look forward to bringing you this additional information on Speaking ASQAneese – Part 2.
Good training,
Joe Newbery
Published: 8th June 2023
Copyright © Newbery Consulting 2023. All rights reserved.