Achieving Error-Free Scoring

A long awaited technological advancement in scoring tests and surveys with less than one error in 1.5 million marks!

Error rates as high 0.5% is unacceptable for students taking SAT and other tests. Error rates near 0.01% is the best ever achieved by standardized scoring systems using Optical Mark Recognition (OMR) technologies. AVANTE has pioneered the use of optical imaging technologies with the innovative use of scaling and quantitative pixel measurements to achieve accuracy rates of 0 errors in 1,500,000 marks! This rate was certified by an independent testing laboratory for Federal election applications.

SOLUTIONS FOR FIXING THE COLLEGE BOARD SAT SCORING ERRORS

The 2006 SAT test scoring and grading errors by the tabulation company engaged by the College Board highlights the need for upgrading current scanning technology. The test assessment and grading errors were attributed to moisture induced paper expansion and the inability to resolve lightly marked “bubble marks”.

The inability to resolve lightly marked “bubble marks” and errors due to paper expansion or shrinkage is a chronic industry problem. Combined with paper alignment errors, approximately 1.5% of the SAT tests were tallied incorrectly. The largest scoring and grading error rate was 450 points or 19% on a test of 2400 points or 25% for a 1600 point test.

Interestingly, the same method and error rates also appeared in tabulation of votes in US elections when paper ballots are used. In fact, the Federal Voting System Standards of 2002 specifically mandate less than 1 error in 500,000 marks when the paper ballot is marked correctly (i.e. properly filled “bubble marks”). Systems using discrete sensor/OMR technology struggle to meet such requirements.

AVANTE International Technology, Inc., though, has developed a solution tallying ballots using pixel count and document imaging technology. The AVANTE system achieve less than one error in 1.5 million marks! The same technology has been used successfully for tabulation of surveys and questionnaires besides tallying ballots.

The following compares AVANTE’s Test and Survey Scoring System and the conventional OMR and discrete sensor mark-sense systems:

COMPARATIVE ANALYSIS OF DISCRETE SENSOR/OMR AND PIXEL-IMAGING (AVANTE Patented US 6,892,944; 7,077,313 and other patents pending)
	DISCRETE SENSOR-OMR	IMAGING + PIXEL COUNT
Recognition of filled “bubble mark”	Yes	Yes
Recognition of lightly filled “bubble mark”	Mostly no (Need high threshold)	Yes
Independence on timing tracks	No	Yes
Ability to resolve registration (fiducial) marks	No	Yes
Ability to resolve wrinkled and creased papers	No	Yes
Ability to resolve paper shrinkage/expansion	No	Yes
Recognition of “þ” and “ý” marks	Mostly No or marginal.	Yes
Resolving barcode (type or individual form)	Yes	Yes
Resolving multiple and different pages	Yes	Yes
Resolving random orientation of pages	No	Yes
“Separating” written answers from “bubbles”	No	Yes
Automatic self-check for accuracy	No	Yes
Ability to retrieve individual tests for recount	No	Yes
Speed (Per Scanner)	Up to 10,000 pages/hr-scanner	Up to 6,000 pages/hr-scanner
Accuracy	> 1/1,000 (2006 SAT case: >1/100)	<1/1,500,000 (Federal ITA tested)

The discrete Optical Mark Recognition (OMR) sensor technology relies on completed documents passing through the scanner “exactly” straight. The width marking positions must line up “correctly” and timing must be exact for reading the length of the paper. With the use of advanced document imaging technology, AVANTE uses fiducial markings to scale for paper orientation and expansion/shrinkage variations. The quantitative use of counting pixels in each “bubble mark” also provides automatic self-checking for possible light marks, smears or errors. The following table is a summary of the differences between traditional discrete sensor OMRs and AVANTE’s imaging with quantitative pixel counting technology.

Comparative Analysis of the Traditional Discrete Sensor Based “OMR” Tabulation System And Pixel Based Optical Imaging Tabulation System
	Discrete Sensor Based “OMR” System	Pixel Based Optical Imaging System (AVANTE patented innovation)
Ease of Audit	Answers are rated by a pre-set threshold of “YES” or “NO” as the forms are scanned. No automatic audit can be done without complete re-scanning with another setting. No electronic images of the forms are kept. Original paper can be used as a last resort.	Forms are first imaged and stored. Actual pixels in each of the marked answers are measured. “YES” or “NO” is set at 20% default. Higher or lower settings are used for automatic audit for any possible errors. Electronic image may be pulled for any specific form for manual audit.
Costs of forms and readers	Precision printed forms cost more. Specialized form readers are more expensive.	Standard office papers are fine. Office laser printers with more than 200 dpi print quality is adequate.
Multiple page forms	Barcode identifiers for pages are used and able to handle properly sorted forms.	Barcode identifiers for pages and forms are used and able to handle forms without pre-sorting.
Stacking and alignment	Forms are preferably pre-stacked and aligned. Some ability to handle mixed forms.	Fiducial marks and barcode identifiers eliminate need for pre-sorting. Even wrinkled forms are OK.
Flexibility of forms	Marks must line up in columnar manner to match the sensor columns. Absolute marking positions against the edge of the paper must be maintained or errors result. Precision printed forms must be used.	A set of fiducial marks is printed for relative position of marked space. Standard laser printed paper forms are acceptable.
Accuracy	Standard election applications show 1 in 1,000-10,000 errors. More carefully marked forms get better results. High moisture or extreme dryness may expand or shrink papers to cause substantially more than 1% error rate.	Demonstrated less than 1 error in 1,500,000 marks. Only 20% volume need to be filled to correctly register. Patented incorporation of fiducial markers and automatic scaling to correct any paper shrinkage or expansion including folding marks, creases and wrinkled papers.