Short answer
Implement a multi-faceted approach to quality assurance, ensuring that both efficiency and thoroughness are considered, especially in critical areas like financial auditing.
- Field
- Commercial Production
- Source
- Journal of Business & Economics Research (JBER) (2011)
- Method
- Comparative analysis
- Evidence
- Strong effect
Desk reviews, while more economical, do not capture the same comprehensive audit quality information as in-depth working paper reviews. This commercial production research insight is drawn from a 2011 study published in Journal of Business & Economics Research (JBER). Using Comparative analysis, researchers explored how this design variable affects real-world outcomes. The key design takeaway: Implement a multi-faceted approach to quality assurance, ensuring that both efficiency and thoroughness are considered, especially in critical areas like financial auditing.
Desk Reviews Inadequately Assess Audit Quality Compared to Working Paper Reviews
Desk reviews, while more economical, do not capture the same comprehensive audit quality information as in-depth working paper reviews.
Journal of Business & Economics Research (JBER) · 2011
Key Findings
- 01Desk reviews and quality control reviews do not measure the same constructs of audit quality.
- 02Desk reviews appear to measure industry-specific knowledge, which is only one component of overall audit quality.
- 03Some audits that passed the desk review were subsequently graded as low quality by the working paper review.
Application
Design takeaway
Implement a multi-faceted approach to quality assurance, ensuring that both efficiency and thoroughness are considered, especially in critical areas like financial auditing.
How to apply
When designing quality control processes for any service or product, consider the trade-offs between speed/cost and the comprehensiveness of the evaluation. For high-stakes outcomes, a more in-depth review is likely necessary.
Project actions
- 01When evaluating a design, consider if your chosen testing method truly reflects the overall quality or just a surface-level aspect.
- 02Think about what 'quality' means in your specific design context and how best to measure it comprehensively.
Method & Evidence
Variables
Strengths & Limitations
Strengths
- +Direct comparison of two distinct review methodologies.
- +Focus on a specific, regulated industry where quality is paramount.
Limitations
The specific context of public sector audits might not directly translate to all design projects. The definition of 'quality' can be subjective.
Reliability & validity
The study's validity relies on the assumption that QCRs of working papers are a more accurate measure of audit quality than desk reviews. Reliability would depend on the consistency of scoring between reviewers.
Think critically
If desk reviews are faster and cheaper, why would organizations continue to perform more expensive working paper reviews if they don't yield different information?
Design Principles
"The depth of a review process should be commensurate with the criticality of the outcome being assessed."
This finding is crucial for organizations that rely on audit quality assessments for regulatory compliance or internal assurance. It suggests that a sole reliance on streamlined review processes might overlook significant quality issues, potentially impacting financial reporting integrity and stakeholder trust.
What This Means for Your Design
Checking only the final report of a project isn't enough to know if the whole project was done well; you need to look at the detailed steps taken.
How to use in your project
- 1.Reference this study when discussing the limitations of superficial testing or the importance of thorough evaluation in your design process.
Add to My Project
Quick Cite
Paragraph starter
This research highlights that superficial checks, such as desk reviews in auditing, may not accurately reflect the true quality of a service or product. Similarly, in a design project, a quick visual inspection might miss underlying functional flaws that a more thorough testing procedure would reveal. Therefore, it is crucial to select evaluation methods that are sufficiently detailed to capture the essential aspects of quality relevant to the design's intended function and user experience.
Source
Journal of Business & Economics Research (JBER)
A Comparative Analysis Of Audit Service Supply Using Desk And Working Paper Reviews
journal · 2011
View sourceQuestions About This Research
- What does the research say about desk reviews inadequately assess audit quality compared to working paper reviews?
- Implement a multi-faceted approach to quality assurance, ensuring that both efficiency and thoroughness are considered, especially in critical areas like financial auditing. Evidence: Journal of Business & Economics Research (JBER) (2011).
- Why does "Desk Reviews Inadequately Assess Audit Quality Compared to Working Paper Reviews" matter for design?
- This finding is crucial for organizations that rely on audit quality assessments for regulatory compliance or internal assurance. It suggests that a sole reliance on streamlined review processes might overlook significant quality issues, potentially impacting financial reporting integrity and stakeholder trust.
- How can designers apply this research?
- Implement a multi-faceted approach to quality assurance, ensuring that both efficiency and thoroughness are considered, especially in critical areas like financial auditing.
- What were the main findings?
- Desk reviews and quality control reviews do not measure the same constructs of audit quality.. Desk reviews appear to measure industry-specific knowledge, which is only one component of overall audit quality.. Some audits that passed the desk review were subsequently graded as low quality by the working paper review.
- What research method was used?
- Comparative analysis.
- How strong is the evidence?
- Evidence strength is rated Strong effect, based on a 2011 journal from Journal of Business & Economics Research (JBER).
- What should I do differently in my next project?
- When designing quality control processes for any service or product, consider the trade-offs between speed/cost and the comprehensiveness of the evaluation. For high-stakes outcomes, a more in-depth review is likely necessary.
- What are the limitations?
- The study focused specifically on audits of Texas school districts, which may limit generalizability to other sectors or jurisdictions. The definition and measurement of 'audit quality' itself can be complex and subject to interpretation.