Psychometric Services
Pubrica provides psychometric services for certification bodies, licensure programmes and other organisations that run assessments. Our psychometricians carry out item analysis, test blueprint design, standard setting, equating across forms and job task analysis. They answer the statistical questions an assessment programme must be able to defend: which items discriminate, how the cutscore was set, whether forms are comparable and whether the blueprint reflects the role. Each deliverable includes the statistical working behind it.
- Item analysis: difficulty, discrimination, distractor performance and reliability, so that weak items are retired on evidence
- Test blueprint design, which is the basis of content validity
- Standard setting and cutscores by an established method, fully documented, so that the pass mark can be defended
- Equating across forms, so that a candidate taking a harder form is not disadvantaged
- Job task analysis where a certification must show that the test measures what the role requires
Item and test analysisIncluded
Blueprint design and job task analysisIncluded
Standard setting and cutscoresIncluded
Equating across formsIncluded
Item bank software and delivery platformYour platform
Blueprint → items → analysed → standard set → defensible
- Blueprint developed before items are written
- Items retired on evidence
- Standard set by an established method
- Forms equated rather than assumed comparable
- Full documentation in case of challenge
Deliverables
Four Components of Every Assessment Programme
Every programme covers four components. The decisions a test is used to make determine how defensible it must be.
From the analysisItem 14 seems too easy; remove it.Item 14: p=0.94, point-biserial 0.06, distractor C selected by no candidate. The item barely discriminates and one option does not function. We recommend retiring it, or rewriting option C if the item is kept.
From the blueprintWhen items are written before the blueprint, content validity can only be established retrospectively, and accreditors question this first.
From the standard settingThe pass mark is 70%.Modified Angoff with twelve subject matter experts across two rounds, cutscore 63.4% with the standard error reported and the panel composition documented. This cutscore can be defended if challenged.
From the equatingWithout equating, candidates told that “the pass mark is the same” have in fact faced different standards on different forms. Challenges on this basis are likely to succeed.
The samples shown are illustrative and anonymised. Hover over a sample to replay it.
Compliance and Standards
Psychometric Testing Standards and Methods
Where an accreditor sets requirements, those requirements take precedence. The following are the established methods we apply.
Standards for Educational and Psychological Testing (AERA, APA, NCME)
The reference standards for validity, reliability, fairness and documentation in testing.
Standard setting and analysis methods (Angoff, bookmark, CTT and IRT)
The established approaches to item analysis, standard setting and equating, selected to suit each programme.
Certification accreditation (programme requirements)
Where a certification is accredited, the accreditor sets expectations on job task analysis, blueprint, standard setting and documentation.
Quality management (cert. H3CR9TCU20251212INDQM10N)
Our certified system for briefing, performing, reviewing and releasing project work.
Responsible Use of AI
Our Approach to AI Use in Psychometric Analysis
Psychometric software will return fit statistics for almost any model. Whether an instrument measures what it claims requires a validity argument built by a specialist who understands the construct, and COSMIN provides the structure for that argument.
Why We Do Not Publish an AI Accuracy Figure
Accuracy figures for AI tools are usually reported by the companies that sell them.
Pubrica therefore does not quote a self-generated accuracy figure, because it would carry the same conflict of interest. Instead, we tell you which tool was used on your project, at which stage, and what a specialist changed afterwards.
Methodology
How Our Psychometric Process Works
We begin with the purpose of the assessment, because a classroom quiz and a licensure examination require very different levels of rigour.
Purpose and stakes
We establish what the assessment is used to decide, who is affected and what would happen if a result were challenged. This determines the level of rigour required.
Blueprint or job task analysis
We build the test specification from the domain or, where a certification must demonstrate the link, from an analysis of what the role requires.
Item and test analysis
We analyse difficulty, discrimination, distractor function and reliability, and apply item response theory modelling where appropriate.
Standard setting and equating
We set cutscores by an established method, document the panel and process, and equate forms so that results are comparable.
Our Experts
Your Psychometric Analysis Team
A named team of subject specialists stays with your project from scoping to delivery. Your project coordinator introduces each team member by name before work begins, and the handover records who completed each stage.
Reviews the scientific content for accuracy and confirms that each conclusion is supported by the evidence. Each expert is matched to your field.
Checks that statistical methods, tables and conclusions are consistent with the analysis plan and that the analysis can be reproduced.
Prepares, cleans and documents the data so that every step from raw data to final output can be traced and reproduced.
Independently checks code, tables and outputs against the analysis plan. Discrepancies are logged and resolved before delivery.
Packages
Psychometric Services Packages
Pricing depends on item volume, the number of forms and whether a job task analysis is included.
Item and test analysis
For programmes with live data that has not been analysed
Quoted per projectBased on item and candidate volume
Delivery By scope
- ✓Item difficulty and discrimination
- ✓Distractor performance
- ✓Reliability estimates
- ✓Recommendations to retire, revise or keep each item
- ✓Written report for your board
For most programmes, item analysis offers the quickest and usually the least costly improvement.
Deliverables Request a quoteFull programme support
For certification and licensure programmes
Quoted per projectBased on programme complexity
Delivery By arrangement
- ✓Job task analysis
- ✓Test blueprint design
- ✓Item analysis on a cycle
- ✓Standard setting with documented panel process
- ✓Equating across forms
- ✓Technical documentation for accreditation
With full documentation, this support makes results defensible when a candidate challenges them.
Deliverables Request a quoteSend one test form and its response data and we will analyse it free of charge and report how many items are not discriminating. In most unanalysed tests, the number is higher than expected.
FAQs
Psychometric Services FAQs
If your question is not listed here, ask an expert. No quote is required.
Begin with item analysis on live data. It is relatively inexpensive and usually identifies a number of items that barely discriminate.
The next step is the blueprint, because content validity comes from the test specification and is much harder to establish retrospectively.
A fixed percentage is a number rather than a standard. It does not describe what a minimally competent candidate should be able to do, and it cannot be defended if challenged.
Standard setting by an established method with a documented panel produces a cutscore you can explain, together with its standard error.
Equating is needed if you use more than one form and treat the results as comparable. Without it, candidates on a harder form face a higher standard while being told it is the same.
Challenges on this basis are the most likely to succeed, and equating prevents them.
Job task analysis is used by certification programmes that must demonstrate that the test measures what the role requires. It provides the evidence linking the blueprint to practice.
Accreditors ask for it. Without one, a programme can assert content validity but cannot provide evidence for it.
Related Services
Related Assessment Services
Psychometric analysis provides the statistical evidence that supports an assessment programme.
1THE ASSESSMENTitems and fairness
Assessment and exam reviewReview of item quality and alignment. Bias and fairness reviewSensitivity review and differential item functioning.2THE DOMAIN MEASUREDstandards and objectives
Standards alignmentContent mapped to the applicable framework. Learning design and pedagogyLearning objectives and assessment strategy.Request a Free Item Analysis of One Test Form
The report states how many items are not discriminating and gives difficulty and discrimination statistics for every item on the form.