For blind students seeking education and a career in science, engineering and
mathematics, the calculus has presented a formidable barrier. This is not due to
the intrinsic difficulty of the subject, which is obstacle enough for most
students. The additional hurdle for blind students is the substantial graphical
component of the typical calculus course. There are two major aspects of that
graphical component: primarily, there is the representation of geometrical
objects, especially the graphs of functions: then, there is the presentation of
mathematical formulas as a graphical display.
Under a grant from the National Science Foundation, the Computer
Science Department (CSD) of the College of Staten Island
and the Computer Center for Visually Impaired People (CCVIP) of Baruch College
are developing text materials and providing
an environment to offer blind and visually impaired students technologically assisted access to the graphical content
of calculus. The goal of the project is to equal or exceed
the quality of courses for students with unimpaired vision.
We are installing facilities for reading mathematical text
and graphics directly without the help of sighted readers. It
is only quite recently that any such technology has become practical and affordable for institutions. We can expect that it
will become so for individuals before long.
We wish to bring such a course into being for the students who
need it right now. For that reason, we are trying to base our
system, insofar as possible, on "off-the-shelf" technology and courseware, rather than try to invent much of it.
For basic course content, we are adapting the successful self- paced mastery
course in calculus developed at Carnegie-Mellon University for students of science, engineering and
mathematics, developed by Albert
Blank assisted by Raymond E. Artz and Erik Hardy (1).
This is being supplemented by essential prerequisite materials that visually
impaired students may lack. The original text was oblivious to
the special needs of these students and many small adaptations will
be made. For example, the original text stated in its second paragraph, "The axes divide the coordinate plane into
four quadrants which are traditionally labeled with Roman numerals as shown in Figure1-1."
No further clarification was given.
The use of a course designed for self-pacing is basic to the program. Our
students will work with new multisensory media that present the course
materials and will augment their skills with new language and new techniques of expression for presenting their own work.
They will generally need extra
time, effort and training. Furthermore, thorough mastery
is vital because the calculus is fundamental to most of science and
engineering. Moreover, mastery is important to give the students confidence that
they can become proficient in areas that were hitherto
largely inaccessible to them.
Many of the problems of presentation to students with visual impairments
can now be addressed with existing technology in a multimedia
We have been using the
touch-sensitive Nomad audio tactile tablet (2) to present
graphic images. Consider the example cited above as Figure 1.1 in the calculus textbook. The tactile
version of this purely verbal and
visual statement is produced on a soft plastic
sheet 16.5 inches wide by 12 inches high (3). A tactile grid, each cell measuring 1 inch by 1 inch, is produced in
relief on this sheet. A visual
display of the figure is presented here.
The more important details of the figure are
presented on the graphic as heavier
and in higher relief than the less important ones in order to
give tactile expression to their varying importance. At the bottom margin
of the graphic, at the base of each vertical line, there is a round
button. When any of these buttons are pressed, Nomad voices the
x-coordinate of that line. Similarly, along the left margin there is a column of buttons that voice the
y-coordinates of the horizontal gridlines
attached to them. The axes are the two special saw tooth lines
that feel bumpy to the touch. The x-axis is the horizontal saw tooth
line attached to the button on the left margin that voices, "y =0;" it
feels smoother to the touch when stroked in the positive direction,
left to right, than in the opposite or negative direction. The
y-axis is the vertical saw tooth line attached to the button in the bottom
margin that voices, "x =0." For this axis, the positive direction and
the smoother to the touch is upward. Pressure at the intersection of
the axes will voice, "origin." Pressure at any other point on the x-
axis will voice, "x-axis," etc. Along the upper right margin of the graphic
there is a row of diamond-shaped buttons. A press on one of these
will voice an associated keyword from the text.
Any feature of a graphic can be programmed to
voice three levels of information
in succession. This is especially useful when applied to the ID
box, which is a rectangle consistently placed in the lower right corner of
the figure. At that location, the first level of information gives the figure
number and name. The next level lists features of the graphic that can
then be located through Nomad's search capabilities. The third level
can give any information about the graphic or the lesson that the instructor
desires. Nomad lacks high level editing capabilities but it is easy,
though sometimes tedious, to program.
The grid spacing and button placement on the
graphic plates is maintained for
all the graphs the student will use in the course. The keywords will differ depending on the lesson. The origin and
axes may lie anywhere or even be
located outside the picture frame. The coordinates
of successive grid points on the axes can differ by some other
constant than one. At the same time, the constant structure of graphics
will offer a consistent, familiar environment in which the student
can operate securely.
The hope for the future is that the
process of making a graphic will be
automated so that a blind person can operate interactively at
a work station to create and analyze such a graphic without requiring
the assistance of a sighted person.
a. Optical Character Recognition
OCR programs offer great promise. However, they are not yet completely reliable readers even of straightforward literary text. Moreover, technical print containing formulas is still far beyond their capabilities.
Braille systems for rendering formulas exist and others are in development. The Nemeth code was developed specifically for the purpose of rendering mathematical expressions in Braille. It uses standard six dot Braille and, by virtue of adroitly constructed combinations of Braille characters, is able to represent very complex expressions. The Nemeth system needs to use compound characters to represent many of the symbols that are single characters on the keyboard. A more significant difficulty is that the code represents the graphical elements of complex mathematical expressions that enable the learner to develop an intuitive grasp on the material entirely by means of six-dot Braille. This has the advantage that it can be embossed by a conventional printer that produces only six dot cells. Since our technology can produce continuous output, we are experimenting with embossing print grouping symbols combined with Nemeth Braille for the grouped elements to find out whether this makes for a better tactile display.
Computer Braille is a six dot system which
represents letters, numerals and
punctuation (including parenthesis, brackets and braces). It
is most useful for communicating between ASCII based and Braille based
devices without the need for a great deal of translation. Computer Braille can be used for mathematical formulas but its
use doesn't make it easier or
faster to understand them.
A hybrid system is being developed by Prof.
John A. Gardner at Oregon State
University under an NSF grant in collaboration with Prof. Norberto Salinas,
University of Kansas. This is the Dots+ system which combines eight dot Braille with tactile display of some
of the graphical elements in
technical formulas. With eight dots per Braille cell.
a true one-to-one match could be made between Braille cells and ASCII's
eight bit bytes. This could be even more useful for blind programmers
than computer Braille. If the Dots+ system receives general acceptance, we plan to incorporate it in
our program. It is important to realize that our graphics are audio
captioned as well as Braille captioned. For now, since we are only using terse captions Grade 1 Braille is all we need.
c. Voice Presentation
Until we are able to install more advanced methods of presenting technical text, we are simply using the time-honored method of preparing audio cassettes made by a trained reader. We plan to install this also on CD-ROM or DVD in the form of .wav files. By use of a large touch screen we should, within a few years, also produce audio feedback for the tactile graphics using .wav files. This approach will enable us to present the entirety of our material as a package of tactile plates and a laser disk.
The application of voice synthesis to read
ASCII text files is a most
promising method for the future. In our multimedia
laboratory, we propose to install ASTER, T.V. Raman's program (3) when it becomes available for use on a PC.
ASTER has great parsing and
expressive capabilities. It
provides an auditory rendering of the structure and
content of mathematical formula in ways analogous to a graphical display. ASTER has excellent hypertext facilities
that permit sophisticated random
search for information. These capabilities
exceed those of a trained mathematical reader, as demonstrated by cassette tapes prepared by Recording for the Blind
and Dyslexic, Princeton, N.J. ASTER reads technical ASCII files written
with the LATEX macro package for the mathematical typesetting language, TEX. The combination of LATEX and ASTER has
a special advantage: it is possible to use a command set that
expresses the semantic content of a symbol as well as its typographical form. For example, the symbol (a, b) for an
ordered pair of entities is used in mathematics in many different
contexts and interpretations. If it were used to represent the coordinates of a point in a plane, say, it would
be possible to use a special LATEX macro that would cause
the symbol to be printed as usual but cause ASTER to speak,
"point a comma b".
As yet, ASTER requires substantial hardware and software resources that would be available only in an institutional setting. Until ASTER is installed, mathematical formulas can be written out in English for synthesis by screen reading programs. For example, the formula
(a+b) / (c+d)
could be voiced as "fraction a
plus b over c plus d", where
the extra spaces are to be read as pauses.
Work executed in LATEX could be printed in
typeset form by the student for
submission to the instructor or the LATEX source file could
be viewed in that form on the instructor's screen. With appropriate
software and a Braille printer, the student's work could be saved in hard copy for future use. Whenever ASTER becomes
available, the student will be able to review his work with the
assistance of ASTER's search facilities. Without ASTER, we would expect that audible review could be done by standard
screen reading systems
"trained" to read the LATEX commands. For example,
"$ \sin x $," would be voiced as, "sine of x."
2. Available from Humanware, Inc. Loomis,
3. Information Technology and Disabilities,
v.1, no.4, November 1994. www.rit.edu/~easi/itd/itdv01n4/article2.html.
We are grateful to Julio C. Perez for
reviewing the section on Braille and
contributing his knowledge.
In addition, we'd like to thank Jen Ciaccio for her hard work in helping to
publicize our efforts, and Dr. Bernie Domanski for helping with the web site
design and development.
In addition, we'd like to thank Jen Ciaccio for her hard work in helping to publicize our efforts, and Dr. Bernie Domanski for helping with the web site design and development.
This work was supported by the National
Science Foundation, Experimental
Projects in Human Resources and Education, Grant No. HRD-9450166 (1994), "Multisensory Calculus for
Teaching Students with Visual
Impairments," and HRD-9906115 (1999), "Multisensory Calculus for Visually Impaired
Albert A. Blank, Professor
Karen Luxton Gourgey, Director
Initial Web Site Development
Questions, comments and constructive suggestions are welcome and encouraged.
Last Update: Saturday, September 22, 2001